cpython-lru_cache-weakref: Lib/tarfile.py annotate

annotate Lib/tarfile.py @ 79691:05c8928362ee

Merge __next__ method link fixes with 3.3.

author	Ezio Melotti <ezio.melotti@gmail.com>
date	Fri, 12 Oct 2012 13:46:37 +0300
parents	15949f34e08f
children	2d887691a99b

rev	line source
59522 fcfe324663a6 convert shebang lines: python -> python3 Benjamin Peterson <benjamin@python.org> parents: 59304 diff changeset	1 #!/usr/bin/env python3
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2 #-------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	3 # tarfile.py
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	4 #-------------------------------------------------------------------
43760 e90065e17212 Disabled test_sys_path_with_unicode for now Christian Heimes <christian@cheimes.de> parents: 43577 diff changeset	5 # Copyright (C) 2002 Lars Gustaebel <lars@gustaebel.de>
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	6 # All rights reserved.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	7 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	8 # Permission is hereby granted, free of charge, to any person
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	9 # obtaining a copy of this software and associated documentation
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	10 # files (the "Software"), to deal in the Software without
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	11 # restriction, including without limitation the rights to use,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	12 # copy, modify, merge, publish, distribute, sublicense, and/or sell
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	13 # copies of the Software, and to permit persons to whom the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	14 # Software is furnished to do so, subject to the following
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	15 # conditions:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	16 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	17 # The above copyright notice and this permission notice shall be
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	18 # included in all copies or substantial portions of the Software.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	19 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	20 # THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	21 # EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	22 # OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	23 # NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	24 # HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	25 # WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	26 # FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	27 # OTHER DEALINGS IN THE SOFTWARE.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	28 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	29 """Read from and write to tar format archives.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	30 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	31
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	32 version = "0.9.0"
43777 0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch. Guido van Rossum <guido@python.org> parents: 43760 diff changeset	33 __author__ = "Lars Gust\u00e4bel (lars@gustaebel.de)"
71540 3e26c9033306 Fix closes Issue11439 Remove the SVN keywords from the code as it is no longer applicable in hg. Patch Contributed by Neil Muller. Senthil Kumaran <senthil@uthcode.com> parents: 68257 diff changeset	34 __date__ = "$Date: 2011-02-25 17:42:01 +0200 (Fri, 25 Feb 2011) $"
3e26c9033306 Fix closes Issue11439 Remove the SVN keywords from the code as it is no longer applicable in hg. Patch Contributed by Neil Muller. Senthil Kumaran <senthil@uthcode.com> parents: 68257 diff changeset	35 __cvsid__ = "$Id: tarfile.py 88586 2011-02-25 15:42:01Z marc-andre.lemburg $"
43777 0d462d789b18 Merging the py3k-pep3137 branch back into the py3k branch. Guido van Rossum <guido@python.org> parents: 43760 diff changeset	36 __credits__ = "Gustavo Niemeyer, Niels Gust\u00e4bel, Richard Townsend."
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	37
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	38 #---------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	39 # Imports
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	40 #---------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	41 import sys
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	42 import os
74241 120a79b8bb11 use io.SEEK_* constants instead of os.SEEK_* where an IO stream is seeked, leaving the os.SEEK_* constants only for os.lseek, as documented Eli Bendersky <eliben@gmail.com> parents: 73927 diff changeset	43 import io
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	44 import shutil
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	45 import stat
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	46 import time
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	47 import struct
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	48 import copy
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	49 import re
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	50
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	51 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	52 import grp, pwd
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	53 except ImportError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	54 grp = pwd = None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	55
62898 5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	56 # os.symlink on Windows prior to 6.0 raises NotImplementedError
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	57 symlink_exception = (AttributeError, NotImplementedError)
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	58 try:
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	59 # WindowsError (1314) will be raised if the caller does not hold the
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	60 # SeCreateSymbolicLinkPrivilege privilege
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	61 symlink_exception += (WindowsError,)
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	62 except NameError:
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	63 pass
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	64
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	65 # from tarfile import *
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	66 __all__ = ["TarFile", "TarInfo", "is_tarfile", "TarError"]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	67
44098 0889daf6d0d9 #1535: rename __builtin__ module to builtins. Georg Brandl <georg@python.org> parents: 43947 diff changeset	68 from builtins import open as _open # Since 'open' is TarFile.open
39649 7496f94754ff Fix fallout from Anna's file -> open changes. Guido van Rossum <guido@python.org> parents: 39647 diff changeset	69
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	70 #---------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	71 # tar constants
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	72 #---------------------------------------------------------
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	73 NUL = b"\0" # the null character
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	74 BLOCKSIZE = 512 # length of processing blocks
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	75 RECORDSIZE = BLOCKSIZE * 20 # length of records
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	76 GNU_MAGIC = b"ustar \0" # magic gnu tar string
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	77 POSIX_MAGIC = b"ustar\x0000" # magic posix tar string
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	78
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	79 LENGTH_NAME = 100 # maximum length of a filename
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	80 LENGTH_LINK = 100 # maximum length of a linkname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	81 LENGTH_PREFIX = 155 # maximum length of the prefix field
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	82
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	83 REGTYPE = b"0" # regular file
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	84 AREGTYPE = b"\0" # regular file
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	85 LNKTYPE = b"1" # link (inside tarfile)
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	86 SYMTYPE = b"2" # symbolic link
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	87 CHRTYPE = b"3" # character special device
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	88 BLKTYPE = b"4" # block special device
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	89 DIRTYPE = b"5" # directory
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	90 FIFOTYPE = b"6" # fifo special device
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	91 CONTTYPE = b"7" # contiguous file
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	92
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	93 GNUTYPE_LONGNAME = b"L" # GNU tar longname
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	94 GNUTYPE_LONGLINK = b"K" # GNU tar longlink
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	95 GNUTYPE_SPARSE = b"S" # GNU tar sparse file
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	96
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	97 XHDTYPE = b"x" # POSIX.1-2001 extended header
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	98 XGLTYPE = b"g" # POSIX.1-2001 global header
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	99 SOLARIS_XHDTYPE = b"X" # Solaris extended header
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	100
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	101 USTAR_FORMAT = 0 # POSIX.1-1988 (ustar) format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	102 GNU_FORMAT = 1 # GNU tar format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	103 PAX_FORMAT = 2 # POSIX.1-2001 (pax) format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	104 DEFAULT_FORMAT = GNU_FORMAT
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	105
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	106 #---------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	107 # tarfile constants
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	108 #---------------------------------------------------------
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	109 # File types that tarfile supports:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	110 SUPPORTED_TYPES = (REGTYPE, AREGTYPE, LNKTYPE,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	111 SYMTYPE, DIRTYPE, FIFOTYPE,
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	112 CONTTYPE, CHRTYPE, BLKTYPE,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	113 GNUTYPE_LONGNAME, GNUTYPE_LONGLINK,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	114 GNUTYPE_SPARSE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	115
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	116 # File types that will be treated as a regular file.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	117 REGULAR_TYPES = (REGTYPE, AREGTYPE,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	118 CONTTYPE, GNUTYPE_SPARSE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	119
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	120 # File types that are part of the GNU tar format.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	121 GNU_TYPES = (GNUTYPE_LONGNAME, GNUTYPE_LONGLINK,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	122 GNUTYPE_SPARSE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	123
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	124 # Fields from a pax header that override a TarInfo attribute.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	125 PAX_FIELDS = ("path", "linkpath", "size", "mtime",
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	126 "uid", "gid", "uname", "gname")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	127
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	128 # Fields from a pax header that are affected by hdrcharset.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	129 PAX_NAME_FIELDS = {"path", "linkpath", "uname", "gname"}
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	130
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	131 # Fields in a pax header that are numbers, all other fields
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	132 # are treated as strings.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	133 PAX_NUMBER_FIELDS = {
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	134 "atime": float,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	135 "ctime": float,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	136 "mtime": float,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	137 "uid": int,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	138 "gid": int,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	139 "size": int
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	140 }
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	141
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	142 #---------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	143 # Bits used in the mode field, values in octal.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	144 #---------------------------------------------------------
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	145 S_IFLNK = 0o120000 # symbolic link
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	146 S_IFREG = 0o100000 # regular file
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	147 S_IFBLK = 0o060000 # block device
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	148 S_IFDIR = 0o040000 # directory
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	149 S_IFCHR = 0o020000 # character device
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	150 S_IFIFO = 0o010000 # fifo
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	151
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	152 TSUID = 0o4000 # set UID on execution
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	153 TSGID = 0o2000 # set GID on execution
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	154 TSVTX = 0o1000 # reserved
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	155
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	156 TUREAD = 0o400 # read by owner
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	157 TUWRITE = 0o200 # write by owner
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	158 TUEXEC = 0o100 # execute/search by owner
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	159 TGREAD = 0o040 # read by group
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	160 TGWRITE = 0o020 # write by group
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	161 TGEXEC = 0o010 # execute/search by group
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	162 TOREAD = 0o004 # read by other
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	163 TOWRITE = 0o002 # write by other
229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	164 TOEXEC = 0o001 # execute/search by other
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	165
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	166 #---------------------------------------------------------
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	167 # initialization
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	168 #---------------------------------------------------------
62274 6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	169 if os.name in ("nt", "ce"):
6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	170 ENCODING = "utf-8"
6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	171 else:
6eaa79be0449 Issue #8784: Set tarfile default encoding to 'utf-8' on Windows. Victor Stinner <victor.stinner@haypocalc.com> parents: 62047 diff changeset	172 ENCODING = sys.getfilesystemencoding()
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	173
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	174 #---------------------------------------------------------
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	175 # Some useful functions
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	176 #---------------------------------------------------------
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	177
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	178 def stn(s, length, encoding, errors):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	179 """Convert a string to a null-terminated bytes object.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	180 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	181 s = s.encode(encoding, errors)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	182 return s[:length] + (length - len(s)) * NUL
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	183
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	184 def nts(s, encoding, errors):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	185 """Convert a null-terminated bytes object to a string.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	186 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	187 p = s.find(b"\0")
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	188 if p != -1:
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	189 s = s[:p]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	190 return s.decode(encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	191
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	192 def nti(s):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	193 """Convert a number field to a python number.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	194 """
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	195 # There are two possible encodings for a number field, see
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	196 # itn() below.
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	197 if s[0] in (0o200, 0o377):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	198 n = 0
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	199 for i in range(len(s) - 1):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	200 n <<= 8
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	201 n += s[i + 1]
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	202 if s[0] == 0o377:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	203 n = -(256 ** (len(s) - 1) - n)
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	204 else:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	205 try:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	206 n = int(nts(s, "ascii", "strict") or "0", 8)
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	207 except ValueError:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	208 raise InvalidHeaderError("invalid header")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	209 return n
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	210
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	211 def itn(n, digits=8, format=DEFAULT_FORMAT):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	212 """Convert a python number to a number field.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	213 """
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	214 # POSIX 1003.1-1988 requires numbers to be encoded as a string of
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	215 # octal digits followed by a null-byte, this allows values up to
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	216 # (8**(digits-1))-1. GNU tar allows storing numbers greater than
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	217 # that if necessary. A leading 0o200 or 0o377 byte indicate this
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	218 # particular encoding, the following digits-1 bytes are a big-endian
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	219 # base-256 representation. This allows values up to (256**(digits-1))-1.
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	220 # A 0o200 byte indicates a positive number, a 0o377 byte a negative
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	221 # number.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	222 if 0 <= n < 8 ** (digits - 1):
42941 171b9f15c223 Conform to strict str/bytes distinction. Lars Gustäbel <lars@gustaebel.de> parents: 42691 diff changeset	223 s = bytes("%0*o" % (digits - 1, n), "ascii") + NUL
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	224 elif format == GNU_FORMAT and -256 (digits - 1) <= n < 256 (digits - 1):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	225 if n >= 0:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	226 s = bytearray([0o200])
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	227 else:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	228 s = bytearray([0o377])
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	229 n = 256 ** digits + n
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	230
72923 341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	231 for i in range(digits - 1):
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	232 s.insert(1, n & 0o377)
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	233 n >>= 8
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	234 else:
341008eab87d Issue #13158: Fix decoding and encoding of base-256 number fields in tarfile. Lars Gustäbel <lars@gustaebel.de> parents: 72272 diff changeset	235 raise ValueError("overflow in number field")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	236
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	237 return s
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	238
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	239 def calc_chksums(buf):
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	240 """Calculate the checksum for a member's header by summing up all
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	241 characters except for the chksum field which is treated as if
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	242 it was filled with spaces. According to the GNU tar sources,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	243 some tars (Sun and NeXT) calculate chksum with signed char,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	244 which will be different if there are chars in the buffer with
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	245 the high bit set. So we calculate two checksums, unsigned and
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	246 signed.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	247 """
77015 c62fa6892424 Issue #13031: Small speed-up for tarfile when unzipping tarfiles. Ross Lagerwall <rosslagerwall@gmail.com> parents: 76949 diff changeset	248 unsigned_chksum = 256 + sum(struct.unpack_from("148B8x356B", buf))
c62fa6892424 Issue #13031: Small speed-up for tarfile when unzipping tarfiles. Ross Lagerwall <rosslagerwall@gmail.com> parents: 76949 diff changeset	249 signed_chksum = 256 + sum(struct.unpack_from("148b8x356b", buf))
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	250 return unsigned_chksum, signed_chksum
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	251
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	252 def copyfileobj(src, dst, length=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	253 """Copy length bytes from fileobj src to fileobj dst.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	254 If length is None, copy the entire content.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	255 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	256 if length == 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	257 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	258 if length is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	259 shutil.copyfileobj(src, dst)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	260 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	261
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	262 BUFSIZE = 16 * 1024
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	263 blocks, remainder = divmod(length, BUFSIZE)
41642 a6eb6acfe04a Merged revisions 55007-55179 via svnmerge from Guido van Rossum <guido@python.org> parents: 41575 diff changeset	264 for b in range(blocks):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	265 buf = src.read(BUFSIZE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	266 if len(buf) < BUFSIZE:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	267 raise IOError("end of file reached")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	268 dst.write(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	269
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	270 if remainder != 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	271 buf = src.read(remainder)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	272 if len(buf) < remainder:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	273 raise IOError("end of file reached")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	274 dst.write(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	275 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	276
76949 492e6c6a01bb #14807: move undocumented tarfile.filemode() to stat.filemode(). Add tarfile.filemode alias with deprecation warning. Giampaolo Rodola' <g.rodola@gmail.com> parents: 76918 diff changeset	277 def filemode(mode):
492e6c6a01bb #14807: move undocumented tarfile.filemode() to stat.filemode(). Add tarfile.filemode alias with deprecation warning. Giampaolo Rodola' <g.rodola@gmail.com> parents: 76918 diff changeset	278 """Deprecated in this location; use stat.filemode."""
492e6c6a01bb #14807: move undocumented tarfile.filemode() to stat.filemode(). Add tarfile.filemode alias with deprecation warning. Giampaolo Rodola' <g.rodola@gmail.com> parents: 76918 diff changeset	279 import warnings
492e6c6a01bb #14807: move undocumented tarfile.filemode() to stat.filemode(). Add tarfile.filemode alias with deprecation warning. Giampaolo Rodola' <g.rodola@gmail.com> parents: 76918 diff changeset	280 warnings.warn("deprecated in favor of stat.filemode",
492e6c6a01bb #14807: move undocumented tarfile.filemode() to stat.filemode(). Add tarfile.filemode alias with deprecation warning. Giampaolo Rodola' <g.rodola@gmail.com> parents: 76918 diff changeset	281 DeprecationWarning, 2)
492e6c6a01bb #14807: move undocumented tarfile.filemode() to stat.filemode(). Add tarfile.filemode alias with deprecation warning. Giampaolo Rodola' <g.rodola@gmail.com> parents: 76918 diff changeset	282 return stat.filemode(mode)
33810 3ff252715d93 [Patch #1043972, for bug #1017553] filemode() returns an incorrect value for the mode 07111 Andrew M. Kuchling <amk@amk.ca> parents: 33507 diff changeset	283
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	284
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	285 class TarError(Exception):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	286 """Base exception."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	287 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	288 class ExtractError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	289 """General exception for extract errors."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	290 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	291 class ReadError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	292 """Exception for unreadble tar archives."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	293 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	294 class CompressionError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	295 """Exception for unavailable compression methods."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	296 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	297 class StreamError(TarError):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	298 """Exception for unsupported operations on stream-like TarFiles."""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	299 pass
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	300 class HeaderError(TarError):
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	301 """Base exception for header errors."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	302 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	303 class EmptyHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	304 """Exception for empty headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	305 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	306 class TruncatedHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	307 """Exception for truncated headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	308 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	309 class EOFHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	310 """Exception for end of file headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	311 pass
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	312 class InvalidHeaderError(HeaderError):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	313 """Exception for invalid headers."""
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	314 pass
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	315 class SubsequentHeaderError(HeaderError):
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	316 """Exception for missing and invalid extended headers."""
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	317 pass
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	318
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	319 #---------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	320 # internal stream interface
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	321 #---------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	322 class _LowLevelFile:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	323 """Low-level file object. Supports reading and writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	324 It is used instead of a regular file object for streaming
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	325 access.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	326 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	327
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	328 def __init__(self, name, mode):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	329 mode = {
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	330 "r": os.O_RDONLY,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	331 "w": os.O_WRONLY \| os.O_CREAT \| os.O_TRUNC,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	332 }[mode]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	333 if hasattr(os, "O_BINARY"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	334 mode \|= os.O_BINARY
61092 405237b11104 Merged revisions 80618 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57736 diff changeset	335 self.fd = os.open(name, mode, 0o666)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	336
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	337 def close(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	338 os.close(self.fd)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	339
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	340 def read(self, size):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	341 return os.read(self.fd, size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	342
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	343 def write(self, s):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	344 os.write(self.fd, s)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	345
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	346 class _Stream:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	347 """Class that serves as an adapter between TarFile and
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	348 a stream-like object. The stream-like object only
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	349 needs to have a read() or write() method and is accessed
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	350 blockwise. Use of gzip or bzip2 compression is possible.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	351 A stream-like object could be for example: sys.stdin,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	352 sys.stdout, a socket, a tape device etc.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	353
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	354 _Stream is intended to be used only internally.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	355 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	356
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	357 def __init__(self, name, mode, comptype, fileobj, bufsize):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	358 """Construct a _Stream object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	359 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	360 self._extfileobj = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	361 if fileobj is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	362 fileobj = _LowLevelFile(name, mode)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	363 self._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	364
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	365 if comptype == '*':
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	366 # Enable transparent compression detection for the
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	367 # stream interface
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	368 fileobj = _StreamProxy(fileobj)
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	369 comptype = fileobj.getcomptype()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	370
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	371 self.name = name or ""
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	372 self.mode = mode
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	373 self.comptype = comptype
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	374 self.fileobj = fileobj
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	375 self.bufsize = bufsize
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	376 self.buf = b""
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	377 self.pos = 0
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	378 self.closed = False
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	379
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	380 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	381 if comptype == "gz":
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	382 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	383 import zlib
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	384 except ImportError:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	385 raise CompressionError("zlib module is not available")
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	386 self.zlib = zlib
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	387 self.crc = zlib.crc32(b"")
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	388 if mode == "r":
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	389 self._init_read_gz()
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	390 self.exception = zlib.error
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	391 else:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	392 self._init_write_gz()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	393
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	394 elif comptype == "bz2":
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	395 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	396 import bz2
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	397 except ImportError:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	398 raise CompressionError("bz2 module is not available")
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	399 if mode == "r":
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	400 self.dbuf = b""
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	401 self.cmp = bz2.BZ2Decompressor()
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	402 self.exception = IOError
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	403 else:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	404 self.cmp = bz2.BZ2Compressor()
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	405
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	406 elif comptype == "xz":
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	407 try:
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	408 import lzma
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	409 except ImportError:
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	410 raise CompressionError("lzma module is not available")
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	411 if mode == "r":
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	412 self.dbuf = b""
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	413 self.cmp = lzma.LZMADecompressor()
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	414 self.exception = lzma.LZMAError
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	415 else:
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	416 self.cmp = lzma.LZMACompressor()
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	417
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	418 elif comptype != "tar":
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	419 raise CompressionError("unknown compression type %r" % comptype)
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	420
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	421 except:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	422 if not self._extfileobj:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	423 self.fileobj.close()
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	424 self.closed = True
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	425 raise
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	426
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	427 def __del__(self):
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	428 if hasattr(self, "closed") and not self.closed:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	429 self.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	430
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	431 def _init_write_gz(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	432 """Initialize for writing with gzip compression.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	433 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	434 self.cmp = self.zlib.compressobj(9, self.zlib.DEFLATED,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	435 -self.zlib.MAX_WBITS,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	436 self.zlib.DEF_MEM_LEVEL,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	437 0)
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	438 timestamp = struct.pack("<L", int(time.time()))
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	439 self.__write(b"\037\213\010\010" + timestamp + b"\002\377")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	440 if self.name.endswith(".gz"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	441 self.name = self.name[:-3]
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	442 # RFC1952 says we must use ISO-8859-1 for the FNAME field.
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	443 self.__write(self.name.encode("iso-8859-1", "replace") + NUL)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	444
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	445 def write(self, s):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	446 """Write string s to the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	447 """
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	448 if self.comptype == "gz":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	449 self.crc = self.zlib.crc32(s, self.crc)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	450 self.pos += len(s)
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	451 if self.comptype != "tar":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	452 s = self.cmp.compress(s)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	453 self.__write(s)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	454
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	455 def __write(self, s):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	456 """Write string s to the stream if a whole new block
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	457 is ready to be written.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	458 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	459 self.buf += s
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	460 while len(self.buf) > self.bufsize:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	461 self.fileobj.write(self.buf[:self.bufsize])
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	462 self.buf = self.buf[self.bufsize:]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	463
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	464 def close(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	465 """Close the _Stream object. No operation should be
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	466 done on it afterwards.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	467 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	468 if self.closed:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	469 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	470
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	471 if self.mode == "w" and self.comptype != "tar":
33234 ab34980ffa16 Flush bz2 data even if nothing had been written so far. Fixes #1013882. Martin v. Löwis <martin@v.loewis.de> parents: 33208 diff changeset	472 self.buf += self.cmp.flush()
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	473
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	474 if self.mode == "w" and self.buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	475 self.fileobj.write(self.buf)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	476 self.buf = b""
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	477 if self.comptype == "gz":
39442 654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	478 # The native zlib crc is an unsigned 32-bit integer, but
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	479 # the Python wrapper implicitly casts that to a signed C
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	480 # long. So, on a 32-bit box self.crc may "look negative",
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	481 # while the same crc on a 64-bit box may "look positive".
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	482 # To avoid irksome warnings from the `struct` module, force
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	483 # it to look positive on all boxes.
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	484 self.fileobj.write(struct.pack("<L", self.crc & 0xffffffff))
1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	485 self.fileobj.write(struct.pack("<L", self.pos & 0xffffFFFF))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	486
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	487 if not self._extfileobj:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	488 self.fileobj.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	489
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	490 self.closed = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	491
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	492 def _init_read_gz(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	493 """Initialize for reading a gzip compressed fileobj.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	494 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	495 self.cmp = self.zlib.decompressobj(-self.zlib.MAX_WBITS)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	496 self.dbuf = b""
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	497
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	498 # taken from gzip.GzipFile with some alterations
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	499 if self.__read(2) != b"\037\213":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	500 raise ReadError("not a gzip file")
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	501 if self.__read(1) != b"\010":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	502 raise CompressionError("unsupported compression method")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	503
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	504 flag = ord(self.__read(1))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	505 self.__read(6)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	506
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	507 if flag & 4:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	508 xlen = ord(self.__read(1)) + 256 * ord(self.__read(1))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	509 self.read(xlen)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	510 if flag & 8:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	511 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	512 s = self.__read(1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	513 if not s or s == NUL:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	514 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	515 if flag & 16:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	516 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	517 s = self.__read(1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	518 if not s or s == NUL:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	519 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	520 if flag & 2:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	521 self.__read(2)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	522
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	523 def tell(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	524 """Return the stream's file pointer position.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	525 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	526 return self.pos
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	527
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	528 def seek(self, pos=0):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	529 """Set the stream's file pointer to pos. Negative seeking
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	530 is forbidden.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	531 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	532 if pos - self.pos >= 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	533 blocks, remainder = divmod(pos - self.pos, self.bufsize)
41642 a6eb6acfe04a Merged revisions 55007-55179 via svnmerge from Guido van Rossum <guido@python.org> parents: 41575 diff changeset	534 for i in range(blocks):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	535 self.read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	536 self.read(remainder)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	537 else:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	538 raise StreamError("seeking backwards is not allowed")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	539 return self.pos
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	540
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	541 def read(self, size=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	542 """Return the next size number of bytes from the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	543 If size is not defined, return all bytes of the stream
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	544 up to EOF.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	545 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	546 if size is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	547 t = []
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	548 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	549 buf = self._read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	550 if not buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	551 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	552 t.append(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	553 buf = "".join(t)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	554 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	555 buf = self._read(size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	556 self.pos += len(buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	557 return buf
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	558
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	559 def _read(self, size):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	560 """Return size bytes from the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	561 """
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	562 if self.comptype == "tar":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	563 return self.__read(size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	564
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	565 c = len(self.dbuf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	566 while c < size:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	567 buf = self.__read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	568 if not buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	569 break
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	570 try:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	571 buf = self.cmp.decompress(buf)
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	572 except self.exception:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	573 raise ReadError("invalid compressed data")
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	574 self.dbuf += buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	575 c += len(buf)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	576 buf = self.dbuf[:size]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	577 self.dbuf = self.dbuf[size:]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	578 return buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	579
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	580 def __read(self, size):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	581 """Return size bytes from stream. If internal buffer is empty,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	582 read another block from the stream.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	583 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	584 c = len(self.buf)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	585 while c < size:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	586 buf = self.fileobj.read(self.bufsize)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	587 if not buf:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	588 break
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	589 self.buf += buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	590 c += len(buf)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	591 buf = self.buf[:size]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	592 self.buf = self.buf[size:]
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	593 return buf
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	594 # class _Stream
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	595
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	596 class _StreamProxy(object):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	597 """Small proxy class that enables transparent compression
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	598 detection for the Stream interface (mode 'r\|*').
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	599 """
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	600
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	601 def __init__(self, fileobj):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	602 self.fileobj = fileobj
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	603 self.buf = self.fileobj.read(BLOCKSIZE)
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	604
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	605 def read(self, size):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	606 self.read = self.fileobj.read
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	607 return self.buf
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	608
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	609 def getcomptype(self):
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	610 if self.buf.startswith(b"\x1f\x8b\x08"):
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	611 return "gz"
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	612 elif self.buf[0:3] == b"BZh" and self.buf[4:10] == b"1AY&SY":
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	613 return "bz2"
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	614 elif self.buf.startswith((b"\x5d\x00\x00\x80", b"\xfd7zXZ")):
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	615 return "xz"
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	616 else:
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	617 return "tar"
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	618
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	619 def close(self):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	620 self.fileobj.close()
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	621 # class StreamProxy
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	622
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	623 #------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	624 # Extraction file object
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	625 #------------------------
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	626 class _FileInFile(object):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	627 """A thin wrapper around an existing file object that
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	628 provides a part of its data as an individual file
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	629 object.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	630 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	631
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	632 def __init__(self, fileobj, offset, size, blockinfo=None):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	633 self.fileobj = fileobj
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	634 self.offset = offset
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	635 self.size = size
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	636 self.position = 0
76770 254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	637 self.name = getattr(fileobj, "name", None)
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	638 self.closed = False
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	639
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	640 if blockinfo is None:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	641 blockinfo = [(0, size)]
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	642
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	643 # Construct a map with data and zero blocks.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	644 self.map_index = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	645 self.map = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	646 lastpos = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	647 realpos = self.offset
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	648 for offset, size in blockinfo:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	649 if offset > lastpos:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	650 self.map.append((False, lastpos, offset, None))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	651 self.map.append((True, offset, offset + size, realpos))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	652 realpos += size
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	653 lastpos = offset + size
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	654 if lastpos < self.size:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	655 self.map.append((False, lastpos, self.size, None))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	656
76770 254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	657 def flush(self):
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	658 pass
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	659
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	660 def readable(self):
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	661 return True
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	662
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	663 def writable(self):
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	664 return False
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	665
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	666 def seekable(self):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	667 return self.fileobj.seekable()
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	668
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	669 def tell(self):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	670 """Return the current file position.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	671 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	672 return self.position
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	673
76770 254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	674 def seek(self, position, whence=io.SEEK_SET):
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	675 """Seek to a position in the file.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	676 """
76770 254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	677 if whence == io.SEEK_SET:
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	678 self.position = min(max(position, 0), self.size)
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	679 elif whence == io.SEEK_CUR:
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	680 if position < 0:
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	681 self.position = max(self.position + position, 0)
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	682 else:
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	683 self.position = min(self.position + position, self.size)
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	684 elif whence == io.SEEK_END:
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	685 self.position = max(min(self.size + position, self.size), 0)
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	686 else:
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	687 raise ValueError("Invalid argument")
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	688 return self.position
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	689
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	690 def read(self, size=None):
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	691 """Read data from the file.
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	692 """
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	693 if size is None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	694 size = self.size - self.position
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	695 else:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	696 size = min(size, self.size - self.position)
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	697
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	698 buf = b""
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	699 while size > 0:
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	700 while True:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	701 data, start, stop, offset = self.map[self.map_index]
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	702 if start <= self.position < stop:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	703 break
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	704 else:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	705 self.map_index += 1
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	706 if self.map_index == len(self.map):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	707 self.map_index = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	708 length = min(size, stop - self.position)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	709 if data:
68117 2325fe36aa34 Issue #11224: Improved sparse file read support (r85916) introduced a Lars Gustäbel <lars@gustaebel.de> parents: 67771 diff changeset	710 self.fileobj.seek(offset + (self.position - start))
2325fe36aa34 Issue #11224: Improved sparse file read support (r85916) introduced a Lars Gustäbel <lars@gustaebel.de> parents: 67771 diff changeset	711 buf += self.fileobj.read(length)
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	712 else:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	713 buf += NUL * length
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	714 size -= length
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	715 self.position += length
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	716 return buf
76770 254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	717
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	718 def readinto(self, b):
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	719 buf = self.read(len(b))
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	720 b[:len(buf)] = buf
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	721 return len(buf)
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	722
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	723 def close(self):
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	724 self.closed = True
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	725 #class _FileInFile
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	726
76918 ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	727 class ExFileObject(io.BufferedReader):
ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	728
ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	729 def __init__(self, tarfile, tarinfo):
ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	730 fileobj = _FileInFile(tarfile.fileobj, tarinfo.offset_data,
ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	731 tarinfo.size, tarinfo.sparse)
ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	732 super().__init__(fileobj)
ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	733 #class ExFileObject
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	734
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	735 #------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	736 # Exported Classes
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	737 #------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	738 class TarInfo(object):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	739 """Informational class which holds the details about an
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	740 archive member given by a tar header block.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	741 TarInfo objects are returned by TarFile.getmember(),
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	742 TarFile.getmembers() and TarFile.gettarinfo() and are
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	743 usually created internally.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	744 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	745
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	746 __slots__ = ("name", "mode", "uid", "gid", "size", "mtime",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	747 "chksum", "type", "linkname", "uname", "gname",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	748 "devmajor", "devminor",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	749 "offset", "offset_data", "pax_headers", "sparse",
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	750 "tarfile", "_sparse_structs", "_link_target")
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	751
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	752 def __init__(self, name=""):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	753 """Construct a TarInfo object. name is the optional name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	754 of the member.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	755 """
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	756 self.name = name # member name
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	757 self.mode = 0o644 # file permissions
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	758 self.uid = 0 # user id
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	759 self.gid = 0 # group id
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	760 self.size = 0 # file size
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	761 self.mtime = 0 # modification time
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	762 self.chksum = 0 # header checksum
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	763 self.type = REGTYPE # member type
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	764 self.linkname = "" # link name
65085 6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	765 self.uname = "" # user name
6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	766 self.gname = "" # group name
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	767 self.devmajor = 0 # device major number
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	768 self.devminor = 0 # device minor number
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	769
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	770 self.offset = 0 # the tar header starts here
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	771 self.offset_data = 0 # the file's data starts here
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	772
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	773 self.sparse = None # sparse member information
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	774 self.pax_headers = {} # pax header information
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	775
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	776 # In pax headers the "name" and "linkname" field are called
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	777 # "path" and "linkpath".
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	778 def _getpath(self):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	779 return self.name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	780 def _setpath(self, name):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	781 self.name = name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	782 path = property(_getpath, _setpath)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	783
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	784 def _getlinkpath(self):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	785 return self.linkname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	786 def _setlinkpath(self, linkname):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	787 self.linkname = linkname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	788 linkpath = property(_getlinkpath, _setlinkpath)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	789
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	790 def __repr__(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	791 return "<%s %r at %#x>" % (self.__class__.__name__,self.name,id(self))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	792
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	793 def get_info(self):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	794 """Return the TarInfo's attributes as a dictionary.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	795 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	796 info = {
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	797 "name": self.name,
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	798 "mode": self.mode & 0o7777,
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	799 "uid": self.uid,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	800 "gid": self.gid,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	801 "size": self.size,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	802 "mtime": self.mtime,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	803 "chksum": self.chksum,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	804 "type": self.type,
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	805 "linkname": self.linkname,
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	806 "uname": self.uname,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	807 "gname": self.gname,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	808 "devmajor": self.devmajor,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	809 "devminor": self.devminor
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	810 }
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	811
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	812 if info["type"] == DIRTYPE and not info["name"].endswith("/"):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	813 info["name"] += "/"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	814
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	815 return info
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	816
61278 88466ef1f0cd Issue #8390: tarfile uses surrogateespace as the default error handler Victor Stinner <victor.stinner@haypocalc.com> parents: 61259 diff changeset	817 def tobuf(self, format=DEFAULT_FORMAT, encoding=ENCODING, errors="surrogateescape"):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	818 """Return a tar header as a string of 512 byte blocks.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	819 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	820 info = self.get_info()
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	821
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	822 if format == USTAR_FORMAT:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	823 return self.create_ustar_header(info, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	824 elif format == GNU_FORMAT:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	825 return self.create_gnu_header(info, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	826 elif format == PAX_FORMAT:
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	827 return self.create_pax_header(info, encoding)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	828 else:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	829 raise ValueError("invalid format")
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	830
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	831 def create_ustar_header(self, info, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	832 """Return the object as a ustar header block.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	833 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	834 info["magic"] = POSIX_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	835
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	836 if len(info["linkname"]) > LENGTH_LINK:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	837 raise ValueError("linkname is too long")
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	838
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	839 if len(info["name"]) > LENGTH_NAME:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	840 info["prefix"], info["name"] = self._posix_split_name(info["name"])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	841
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	842 return self._create_header(info, USTAR_FORMAT, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	843
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	844 def create_gnu_header(self, info, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	845 """Return the object as a GNU header block sequence.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	846 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	847 info["magic"] = GNU_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	848
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	849 buf = b""
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	850 if len(info["linkname"]) > LENGTH_LINK:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	851 buf += self._create_gnu_long_header(info["linkname"], GNUTYPE_LONGLINK, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	852
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	853 if len(info["name"]) > LENGTH_NAME:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	854 buf += self._create_gnu_long_header(info["name"], GNUTYPE_LONGNAME, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	855
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	856 return buf + self._create_header(info, GNU_FORMAT, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	857
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	858 def create_pax_header(self, info, encoding):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	859 """Return the object as a ustar header block. If it cannot be
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	860 represented this way, prepend a pax extended header sequence
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	861 with supplement information.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	862 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	863 info["magic"] = POSIX_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	864 pax_headers = self.pax_headers.copy()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	865
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	866 # Test string fields for values that exceed the field length or cannot
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	867 # be represented in ASCII encoding.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	868 for name, hname, length in (
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	869 ("name", "path", LENGTH_NAME), ("linkname", "linkpath", LENGTH_LINK),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	870 ("uname", "uname", 32), ("gname", "gname", 32)):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	871
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	872 if hname in pax_headers:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	873 # The pax header has priority.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	874 continue
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	875
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	876 # Try to encode the string as ASCII.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	877 try:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	878 info[name].encode("ascii", "strict")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	879 except UnicodeEncodeError:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	880 pax_headers[hname] = info[name]
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	881 continue
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	882
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	883 if len(info[name]) > length:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	884 pax_headers[hname] = info[name]
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	885
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	886 # Test number fields for values that exceed the field limit or values
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	887 # that like to be stored as float.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	888 for name, digits in (("uid", 8), ("gid", 8), ("size", 12), ("mtime", 12)):
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	889 if name in pax_headers:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	890 # The pax header has priority. Avoid overflow.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	891 info[name] = 0
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	892 continue
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	893
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	894 val = info[name]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	895 if not 0 <= val < 8 ** (digits - 1) or isinstance(val, float):
41575 beacf40b5960 Rip out all the u"..." literals and calls to unicode(). Guido van Rossum <guido@python.org> parents: 41554 diff changeset	896 pax_headers[name] = str(val)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	897 info[name] = 0
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	898
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	899 # Create a pax extended header if necessary.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	900 if pax_headers:
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	901 buf = self._create_pax_generic_header(pax_headers, XHDTYPE, encoding)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	902 else:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	903 buf = b""
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	904
42691 5bf68fc73923 Fall back to 'ascii' encoding if sys.getfilesystemencoding() returns Lars Gustäbel <lars@gustaebel.de> parents: 42475 diff changeset	905 return buf + self._create_header(info, USTAR_FORMAT, "ascii", "replace")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	906
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	907 @classmethod
42691 5bf68fc73923 Fall back to 'ascii' encoding if sys.getfilesystemencoding() returns Lars Gustäbel <lars@gustaebel.de> parents: 42475 diff changeset	908 def create_pax_global_header(cls, pax_headers):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	909 """Return the object as a pax global header block sequence.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	910 """
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	911 return cls._create_pax_generic_header(pax_headers, XGLTYPE, "utf-8")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	912
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	913 def _posix_split_name(self, name):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	914 """Split a name longer than 100 chars into a prefix
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	915 and a name part.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	916 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	917 prefix = name[:LENGTH_PREFIX + 1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	918 while prefix and prefix[-1] != "/":
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	919 prefix = prefix[:-1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	920
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	921 name = name[len(prefix):]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	922 prefix = prefix[:-1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	923
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	924 if not prefix or len(name) > LENGTH_NAME:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	925 raise ValueError("name is too long")
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	926 return prefix, name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	927
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	928 @staticmethod
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	929 def _create_header(info, format, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	930 """Return a header block. info is a dictionary with file
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	931 information, format must be one of the *_FORMAT constants.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	932 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	933 parts = [
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	934 stn(info.get("name", ""), 100, encoding, errors),
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 42068 diff changeset	935 itn(info.get("mode", 0) & 0o7777, 8, format),
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	936 itn(info.get("uid", 0), 8, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	937 itn(info.get("gid", 0), 8, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	938 itn(info.get("size", 0), 12, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	939 itn(info.get("mtime", 0), 12, format),
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	940 b" ", # checksum field
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	941 info.get("type", REGTYPE),
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	942 stn(info.get("linkname", ""), 100, encoding, errors),
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	943 info.get("magic", POSIX_MAGIC),
65085 6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	944 stn(info.get("uname", ""), 32, encoding, errors),
6af7b4102d77 Merged revisions 85211 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62043 diff changeset	945 stn(info.get("gname", ""), 32, encoding, errors),
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	946 itn(info.get("devmajor", 0), 8, format),
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	947 itn(info.get("devminor", 0), 8, format),
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	948 stn(info.get("prefix", ""), 155, encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	949 ]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	950
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	951 buf = struct.pack("%ds" % BLOCKSIZE, b"".join(parts))
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	952 chksum = calc_chksums(buf[-BLOCKSIZE:])[0]
42941 171b9f15c223 Conform to strict str/bytes distinction. Lars Gustäbel <lars@gustaebel.de> parents: 42691 diff changeset	953 buf = buf[:-364] + bytes("%06o\0" % chksum, "ascii") + buf[-357:]
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	954 return buf
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	955
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	956 @staticmethod
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	957 def _create_payload(payload):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	958 """Return the string payload filled with zero bytes
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	959 up to the next 512 byte border.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	960 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	961 blocks, remainder = divmod(len(payload), BLOCKSIZE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	962 if remainder > 0:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	963 payload += (BLOCKSIZE - remainder) * NUL
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	964 return payload
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	965
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	966 @classmethod
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	967 def _create_gnu_long_header(cls, name, type, encoding, errors):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	968 """Return a GNUTYPE_LONGNAME or GNUTYPE_LONGLINK sequence
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	969 for name.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	970 """
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	971 name = name.encode(encoding, errors) + NUL
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	972
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	973 info = {}
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	974 info["name"] = "././@LongLink"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	975 info["type"] = type
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	976 info["size"] = len(name)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	977 info["magic"] = GNU_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	978
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	979 # create extended header + name blocks.
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	980 return cls._create_header(info, USTAR_FORMAT, encoding, errors) + \
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	981 cls._create_payload(name)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	982
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	983 @classmethod
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	984 def _create_pax_generic_header(cls, pax_headers, type, encoding):
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	985 """Return a POSIX.1-2008 extended or global header sequence
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	986 that contains a list of keyword, value pairs. The values
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	987 must be strings.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	988 """
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	989 # Check if one of the fields contains surrogate characters and thereby
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	990 # forces hdrcharset=BINARY, see _proc_pax() for more information.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	991 binary = False
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	992 for keyword, value in pax_headers.items():
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	993 try:
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	994 value.encode("utf-8", "strict")
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	995 except UnicodeEncodeError:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	996 binary = True
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	997 break
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	998
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	999 records = b""
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1000 if binary:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1001 # Put the hdrcharset field at the beginning of the header.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1002 records += b"21 hdrcharset=BINARY\n"
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1003
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1004 for keyword, value in pax_headers.items():
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	1005 keyword = keyword.encode("utf-8")
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1006 if binary:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1007 # Try to restore the original byte representation of `value'.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1008 # Needless to say, that the encoding must match the string.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1009 value = value.encode(encoding, "surrogateescape")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1010 else:
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	1011 value = value.encode("utf-8")
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1012
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1013 l = len(keyword) + len(value) + 3 # ' ' + '=' + '\n'
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1014 n = p = 0
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1015 while True:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1016 n = l + len(str(p))
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1017 if n == p:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1018 break
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1019 p = n
42941 171b9f15c223 Conform to strict str/bytes distinction. Lars Gustäbel <lars@gustaebel.de> parents: 42691 diff changeset	1020 records += bytes(str(p), "ascii") + b" " + keyword + b"=" + value + b"\n"
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1021
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1022 # We use a hardcoded "././@PaxHeader" name like star does
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1023 # instead of the one that POSIX recommends.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1024 info = {}
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1025 info["name"] = "././@PaxHeader"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1026 info["type"] = type
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1027 info["size"] = len(records)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1028 info["magic"] = POSIX_MAGIC
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1029
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1030 # Create pax header + record blocks.
42691 5bf68fc73923 Fall back to 'ascii' encoding if sys.getfilesystemencoding() returns Lars Gustäbel <lars@gustaebel.de> parents: 42475 diff changeset	1031 return cls._create_header(info, USTAR_FORMAT, "ascii", "replace") + \
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1032 cls._create_payload(records)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1033
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1034 @classmethod
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1035 def frombuf(cls, buf, encoding, errors):
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1036 """Construct a TarInfo object from a 512 byte bytes object.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1037 """
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1038 if len(buf) == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1039 raise EmptyHeaderError("empty header")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1040 if len(buf) != BLOCKSIZE:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1041 raise TruncatedHeaderError("truncated header")
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1042 if buf.count(NUL) == BLOCKSIZE:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1043 raise EOFHeaderError("end of file header")
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1044
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1045 chksum = nti(buf[148:156])
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1046 if chksum not in calc_chksums(buf):
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1047 raise InvalidHeaderError("bad checksum")
32770 a3b1d6c99e46 SF #918101, allow files >= 8 GB using GNU extension Neal Norwitz <nnorwitz@gmail.com> parents: 32769 diff changeset	1048
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1049 obj = cls()
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1050 obj.name = nts(buf[0:100], encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1051 obj.mode = nti(buf[100:108])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1052 obj.uid = nti(buf[108:116])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1053 obj.gid = nti(buf[116:124])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1054 obj.size = nti(buf[124:136])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1055 obj.mtime = nti(buf[136:148])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1056 obj.chksum = chksum
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1057 obj.type = buf[156:157]
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1058 obj.linkname = nts(buf[157:257], encoding, errors)
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1059 obj.uname = nts(buf[265:297], encoding, errors)
29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1060 obj.gname = nts(buf[297:329], encoding, errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1061 obj.devmajor = nti(buf[329:337])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1062 obj.devminor = nti(buf[337:345])
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1063 prefix = nts(buf[345:500], encoding, errors)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1064
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1065 # Old V7 tar format represents a directory as a regular
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1066 # file with a trailing slash.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1067 if obj.type == AREGTYPE and obj.name.endswith("/"):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1068 obj.type = DIRTYPE
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1069
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1070 # The old GNU sparse format occupies some of the unused
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1071 # space in the buffer for up to 4 sparse structures.
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1072 # Save the them for later processing in _proc_sparse().
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1073 if obj.type == GNUTYPE_SPARSE:
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1074 pos = 386
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1075 structs = []
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1076 for i in range(4):
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1077 try:
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1078 offset = nti(buf[pos:pos + 12])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1079 numbytes = nti(buf[pos + 12:pos + 24])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1080 except ValueError:
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1081 break
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1082 structs.append((offset, numbytes))
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1083 pos += 24
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1084 isextended = bool(buf[482])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1085 origsize = nti(buf[483:495])
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1086 obj._sparse_structs = (structs, isextended, origsize)
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1087
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1088 # Remove redundant slashes from directories.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1089 if obj.isdir():
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1090 obj.name = obj.name.rstrip("/")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1091
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1092 # Reconstruct a ustar longname.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1093 if prefix and obj.type not in GNU_TYPES:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1094 obj.name = prefix + "/" + obj.name
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1095 return obj
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1096
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1097 @classmethod
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1098 def fromtarfile(cls, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1099 """Return the next TarInfo object from TarFile object
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1100 tarfile.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1101 """
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1102 buf = tarfile.fileobj.read(BLOCKSIZE)
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1103 obj = cls.frombuf(buf, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1104 obj.offset = tarfile.fileobj.tell() - BLOCKSIZE
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1105 return obj._proc_member(tarfile)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1106
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1107 #--------------------------------------------------------------------------
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1108 # The following are methods that are called depending on the type of a
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1109 # member. The entry point is _proc_member() which can be overridden in a
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1110 # subclass to add custom _proc_() methods. A _proc_() method MUST
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1111 # implement the following
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1112 # operations:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1113 # 1. Set self.offset_data to the position where the data blocks begin,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1114 # if there is data that follows.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1115 # 2. Set tarfile.offset to the position where the next member's header will
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1116 # begin.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1117 # 3. Return self or another valid TarInfo object.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1118 def _proc_member(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1119 """Choose the right processing method depending on
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1120 the type and call it.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1121 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1122 if self.type in (GNUTYPE_LONGNAME, GNUTYPE_LONGLINK):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1123 return self._proc_gnulong(tarfile)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1124 elif self.type == GNUTYPE_SPARSE:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1125 return self._proc_sparse(tarfile)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1126 elif self.type in (XHDTYPE, XGLTYPE, SOLARIS_XHDTYPE):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1127 return self._proc_pax(tarfile)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1128 else:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1129 return self._proc_builtin(tarfile)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1130
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1131 def _proc_builtin(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1132 """Process a builtin type or an unknown type which
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1133 will be treated as a regular file.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1134 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1135 self.offset_data = tarfile.fileobj.tell()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1136 offset = self.offset_data
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1137 if self.isreg() or self.type not in SUPPORTED_TYPES:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1138 # Skip the following data blocks.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1139 offset += self._block(self.size)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1140 tarfile.offset = offset
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1141
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1142 # Patch the TarInfo object with saved global
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1143 # header information.
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1144 self._apply_pax_info(tarfile.pax_headers, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1145
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1146 return self
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1147
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1148 def _proc_gnulong(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1149 """Process the blocks that hold a GNU longname
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1150 or longlink member.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1151 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1152 buf = tarfile.fileobj.read(self._block(self.size))
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1153
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1154 # Fetch the next header and process it.
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1155 try:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1156 next = self.fromtarfile(tarfile)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1157 except HeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1158 raise SubsequentHeaderError("missing or bad subsequent header")
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1159
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1160 # Patch the TarInfo object from the next header with
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1161 # the longname information.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1162 next.offset = self.offset
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1163 if self.type == GNUTYPE_LONGNAME:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1164 next.name = nts(buf, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1165 elif self.type == GNUTYPE_LONGLINK:
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1166 next.linkname = nts(buf, tarfile.encoding, tarfile.errors)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1167
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1168 return next
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1169
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1170 def _proc_sparse(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1171 """Process a GNU sparse header plus extra headers.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1172 """
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1173 # We already collected some sparse structures in frombuf().
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1174 structs, isextended, origsize = self._sparse_structs
72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1175 del self._sparse_structs
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1176
46441 72a7648699d0 Issue #2058: Remove the buf attribute and add __slots__ to the Lars Gustäbel <lars@gustaebel.de> parents: 45192 diff changeset	1177 # Collect sparse structures from extended header blocks.
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1178 while isextended:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1179 buf = tarfile.fileobj.read(BLOCKSIZE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1180 pos = 0
41642 a6eb6acfe04a Merged revisions 55007-55179 via svnmerge from Guido van Rossum <guido@python.org> parents: 41575 diff changeset	1181 for i in range(21):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1182 try:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1183 offset = nti(buf[pos:pos + 12])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1184 numbytes = nti(buf[pos + 12:pos + 24])
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1185 except ValueError:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1186 break
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1187 if offset and numbytes:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1188 structs.append((offset, numbytes))
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1189 pos += 24
42475 29735899ca15 Completed str/unicode unification. Lars Gustäbel <lars@gustaebel.de> parents: 42132 diff changeset	1190 isextended = bool(buf[504])
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1191 self.sparse = structs
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1192
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1193 self.offset_data = tarfile.fileobj.tell()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1194 tarfile.offset = self.offset_data + self._block(self.size)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1195 self.size = origsize
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1196 return self
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1197
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1198 def _proc_pax(self, tarfile):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1199 """Process an extended or global header as described in
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1200 POSIX.1-2008.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1201 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1202 # Read the header information.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1203 buf = tarfile.fileobj.read(self._block(self.size))
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1204
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1205 # A pax header stores supplemental information for either
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1206 # the following file (extended) or all following files
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1207 # (global).
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1208 if self.type == XGLTYPE:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1209 pax_headers = tarfile.pax_headers
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1210 else:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1211 pax_headers = tarfile.pax_headers.copy()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1212
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1213 # Check if the pax header contains a hdrcharset field. This tells us
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1214 # the encoding of the path, linkpath, uname and gname fields. Normally,
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1215 # these fields are UTF-8 encoded but since POSIX.1-2008 tar
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1216 # implementations are allowed to store them as raw binary strings if
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1217 # the translation to UTF-8 fails.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1218 match = re.search(br"\d+ hdrcharset=([^\n]+)\n", buf)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1219 if match is not None:
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	1220 pax_headers["hdrcharset"] = match.group(1).decode("utf-8")
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1221
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1222 # For the time being, we don't care about anything other than "BINARY".
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1223 # The only other value that is currently allowed by the standard is
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1224 # "ISO-IR 10646 2000 UTF-8" in other words UTF-8.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1225 hdrcharset = pax_headers.get("hdrcharset")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1226 if hdrcharset == "BINARY":
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1227 encoding = tarfile.encoding
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1228 else:
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	1229 encoding = "utf-8"
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1230
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1231 # Parse pax header information. A record looks like that:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1232 # "%d %s=%s\n" % (length, keyword, value). length is the size
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1233 # of the complete record including the length field itself and
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1234 # the newline. keyword and value are both UTF-8 encoded strings.
48769 83b2d07b475b #2834: Change re module semantics, so that str and bytes mixing is forbidden, Antoine Pitrou <solipsis@pitrou.net> parents: 48644 diff changeset	1235 regex = re.compile(br"(\d+) ([^=]+)=")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1236 pos = 0
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1237 while True:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1238 match = regex.match(buf, pos)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1239 if not match:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1240 break
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1241
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1242 length, keyword = match.groups()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1243 length = int(length)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1244 value = buf[match.end(2) + 1:match.start(1) + length - 1]
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1245
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	1246 # Normally, we could just use "utf-8" as the encoding and "strict"
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1247 # as the error handler, but we better not take the risk. For
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1248 # example, GNU tar <= 1.23 is known to store filenames it cannot
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1249 # translate to UTF-8 as raw strings (unfortunately without a
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1250 # hdrcharset=BINARY header).
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1251 # We first try the strict standard encoding, and if that fails we
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1252 # fall back on the user's encoding and error handler.
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	1253 keyword = self._decode_pax_field(keyword, "utf-8", "utf-8",
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1254 tarfile.errors)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1255 if keyword in PAX_NAME_FIELDS:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1256 value = self._decode_pax_field(value, encoding, tarfile.encoding,
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1257 tarfile.errors)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1258 else:
68142 be6c38d1817b Normalize the encoding names for Latin-1 and UTF-8 to Marc-André Lemburg <mal@egenix.com> parents: 68117 diff changeset	1259 value = self._decode_pax_field(value, "utf-8", "utf-8",
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1260 tarfile.errors)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1261
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1262 pax_headers[keyword] = value
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1263 pos += length
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1264
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1265 # Fetch the next header.
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1266 try:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1267 next = self.fromtarfile(tarfile)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1268 except HeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1269 raise SubsequentHeaderError("missing or bad subsequent header")
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1270
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1271 # Process GNU sparse information.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1272 if "GNU.sparse.map" in pax_headers:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1273 # GNU extended sparse format version 0.1.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1274 self._proc_gnusparse_01(next, pax_headers)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1275
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1276 elif "GNU.sparse.size" in pax_headers:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1277 # GNU extended sparse format version 0.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1278 self._proc_gnusparse_00(next, pax_headers, buf)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1279
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1280 elif pax_headers.get("GNU.sparse.major") == "1" and pax_headers.get("GNU.sparse.minor") == "0":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1281 # GNU extended sparse format version 1.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1282 self._proc_gnusparse_10(next, pax_headers, tarfile)
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1283
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1284 if self.type in (XHDTYPE, SOLARIS_XHDTYPE):
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1285 # Patch the TarInfo object with the extended header info.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1286 next._apply_pax_info(pax_headers, tarfile.encoding, tarfile.errors)
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1287 next.offset = self.offset
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1288
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1289 if "size" in pax_headers:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1290 # If the extended header replaces the size field,
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1291 # we need to recalculate the offset where the next
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1292 # header starts.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1293 offset = next.offset_data
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1294 if next.isreg() or next.type not in SUPPORTED_TYPES:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1295 offset += next._block(next.size)
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1296 tarfile.offset = offset
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1297
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1298 return next
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1299
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1300 def _proc_gnusparse_00(self, next, pax_headers, buf):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1301 """Process a GNU tar extended sparse header, version 0.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1302 """
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1303 offsets = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1304 for match in re.finditer(br"\d+ GNU.sparse.offset=(\d+)\n", buf):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1305 offsets.append(int(match.group(1)))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1306 numbytes = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1307 for match in re.finditer(br"\d+ GNU.sparse.numbytes=(\d+)\n", buf):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1308 numbytes.append(int(match.group(1)))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1309 next.sparse = list(zip(offsets, numbytes))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1310
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1311 def _proc_gnusparse_01(self, next, pax_headers):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1312 """Process a GNU tar extended sparse header, version 0.1.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1313 """
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1314 sparse = [int(x) for x in pax_headers["GNU.sparse.map"].split(",")]
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1315 next.sparse = list(zip(sparse[::2], sparse[1::2]))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1316
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1317 def _proc_gnusparse_10(self, next, pax_headers, tarfile):
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1318 """Process a GNU tar extended sparse header, version 1.0.
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1319 """
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1320 fields = None
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1321 sparse = []
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1322 buf = tarfile.fileobj.read(BLOCKSIZE)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1323 fields, buf = buf.split(b"\n", 1)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1324 fields = int(fields)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1325 while len(sparse) < fields * 2:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1326 if b"\n" not in buf:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1327 buf += tarfile.fileobj.read(BLOCKSIZE)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1328 number, buf = buf.split(b"\n", 1)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1329 sparse.append(int(number))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1330 next.offset_data = tarfile.fileobj.tell()
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1331 next.sparse = list(zip(sparse[::2], sparse[1::2]))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1332
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1333 def _apply_pax_info(self, pax_headers, encoding, errors):
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1334 """Replace fields with supplemental information from a previous
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1335 pax extended or global header.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1336 """
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1337 for keyword, value in pax_headers.items():
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1338 if keyword == "GNU.sparse.name":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1339 setattr(self, "path", value)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1340 elif keyword == "GNU.sparse.size":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1341 setattr(self, "size", int(value))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1342 elif keyword == "GNU.sparse.realsize":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1343 setattr(self, "size", int(value))
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1344 elif keyword in PAX_FIELDS:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1345 if keyword in PAX_NUMBER_FIELDS:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1346 try:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1347 value = PAX_NUMBER_FIELDS[keyword](value)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1348 except ValueError:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1349 value = 0
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1350 if keyword == "path":
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1351 value = value.rstrip("/")
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1352 setattr(self, keyword, value)
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1353
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1354 self.pax_headers = pax_headers.copy()
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1355
61675 bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1356 def _decode_pax_field(self, value, encoding, fallback_encoding, fallback_errors):
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1357 """Decode a single field from a pax record.
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1358 """
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1359 try:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1360 return value.decode(encoding, "strict")
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1361 except UnicodeDecodeError:
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1362 return value.decode(fallback_encoding, fallback_errors)
bf14b9638c73 Issue #8633: Support for POSIX.1-2008 binary pax headers. Lars Gustäbel <lars@gustaebel.de> parents: 61278 diff changeset	1363
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1364 def _block(self, count):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1365 """Round up a byte count by BLOCKSIZE and return it,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1366 e.g. _block(834) => 1024.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1367 """
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1368 blocks, remainder = divmod(count, BLOCKSIZE)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1369 if remainder:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1370 blocks += 1
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1371 return blocks * BLOCKSIZE
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1372
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1373 def isreg(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1374 return self.type in REGULAR_TYPES
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1375 def isfile(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1376 return self.isreg()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1377 def isdir(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1378 return self.type == DIRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1379 def issym(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1380 return self.type == SYMTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1381 def islnk(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1382 return self.type == LNKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1383 def ischr(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1384 return self.type == CHRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1385 def isblk(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1386 return self.type == BLKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1387 def isfifo(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1388 return self.type == FIFOTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1389 def issparse(self):
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	1390 return self.sparse is not None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1391 def isdev(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1392 return self.type in (CHRTYPE, BLKTYPE, FIFOTYPE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1393 # class TarInfo
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1394
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1395 class TarFile(object):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1396 """The TarFile Class provides an interface to tar archives.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1397 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1398
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1399 debug = 0 # May be set from 0 (no msgs) to 3 (all msgs)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1400
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1401 dereference = False # If true, add content of linked file to the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1402 # tar file, else the link.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1403
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1404 ignore_zeros = False # If true, skips empty or invalid blocks and
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1405 # continues processing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1406
57681 46c229742c64 Merged revisions 76780 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57394 diff changeset	1407 errorlevel = 1 # If 0, fatal errors only appear in debug
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1408 # messages (if debug >= 0). If > 0, errors
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1409 # are passed to the caller as exceptions.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1410
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1411 format = DEFAULT_FORMAT # The format to use when creating an archive.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1412
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1413 encoding = ENCODING # Encoding for 8-bit character strings.
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1414
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1415 errors = None # Error handler for unicode conversion.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1416
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1417 tarinfo = TarInfo # The default TarInfo class to use.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1418
76918 ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	1419 fileobject = ExFileObject # The file-object for extractfile().
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1420
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1421 def __init__(self, name=None, mode="r", fileobj=None, format=None,
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1422 tarinfo=None, dereference=None, ignore_zeros=None, encoding=None,
61278 88466ef1f0cd Issue #8390: tarfile uses surrogateespace as the default error handler Victor Stinner <victor.stinner@haypocalc.com> parents: 61259 diff changeset	1423 errors="surrogateescape", pax_headers=None, debug=None, errorlevel=None):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1424 """Open an (uncompressed) tar archive `name'. `mode' is either 'r' to
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1425 read from an existing archive, 'a' to append data to an existing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1426 file or 'w' to create a new file overwriting an existing one. `mode'
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1427 defaults to 'r'.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1428 If `fileobj' is given, it is used for reading or writing data. If it
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1429 can be determined, `mode' is overridden by `fileobj's mode.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1430 `fileobj' is not closed, when TarFile is closed.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1431 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1432 if len(mode) > 1 or mode not in "raw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1433 raise ValueError("mode must be 'r', 'a' or 'w'")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1434 self.mode = mode
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1435 self._mode = {"r": "rb", "a": "r+b", "w": "wb"}[mode]
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1436
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1437 if not fileobj:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1438 if self.mode == "a" and not os.path.exists(name):
40893 32c4897b5d46 Merged revisions 53623-53858 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40775 diff changeset	1439 # Create nonexistent files in append mode.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1440 self.mode = "w"
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1441 self._mode = "wb"
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1442 fileobj = bltn_open(name, self._mode)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1443 self._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1444 else:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1445 if name is None and hasattr(fileobj, "name"):
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1446 name = fileobj.name
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1447 if hasattr(fileobj, "mode"):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1448 self._mode = fileobj.mode
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1449 self._extfileobj = True
42954 27f0289b6835 Merge the trunk changes in. Breaks socket.ssl for now. Thomas Wouters <thomas@python.org> parents: 42941 diff changeset	1450 self.name = os.path.abspath(name) if name else None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1451 self.fileobj = fileobj
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1452
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1453 # Init attributes.
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1454 if format is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1455 self.format = format
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1456 if tarinfo is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1457 self.tarinfo = tarinfo
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1458 if dereference is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1459 self.dereference = dereference
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1460 if ignore_zeros is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1461 self.ignore_zeros = ignore_zeros
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1462 if encoding is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1463 self.encoding = encoding
61278 88466ef1f0cd Issue #8390: tarfile uses surrogateespace as the default error handler Victor Stinner <victor.stinner@haypocalc.com> parents: 61259 diff changeset	1464 self.errors = errors
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1465
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1466 if pax_headers is not None and self.format == PAX_FORMAT:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1467 self.pax_headers = pax_headers
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1468 else:
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1469 self.pax_headers = {}
9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1470
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1471 if debug is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1472 self.debug = debug
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1473 if errorlevel is not None:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1474 self.errorlevel = errorlevel
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1475
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1476 # Init datastructures.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1477 self.closed = False
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1478 self.members = [] # list of members as TarInfo objects
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1479 self._loaded = False # flag if all members have been read
44104 ff1766eb2fce Merged revisions 59259-59274 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44098 diff changeset	1480 self.offset = self.fileobj.tell()
ff1766eb2fce Merged revisions 59259-59274 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44098 diff changeset	1481 # current position in the archive file
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1482 self.inodes = {} # dictionary caching the inodes of
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1483 # archive members already added
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1484
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1485 try:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1486 if self.mode == "r":
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1487 self.firstmember = None
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1488 self.firstmember = self.next()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1489
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1490 if self.mode == "a":
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1491 # Move to the end of the archive,
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1492 # before the first empty block.
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1493 while True:
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1494 self.fileobj.seek(self.offset)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1495 try:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1496 tarinfo = self.tarinfo.fromtarfile(self)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1497 self.members.append(tarinfo)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1498 except EOFHeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1499 self.fileobj.seek(self.offset)
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1500 break
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1501 except HeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1502 raise ReadError(str(e))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1503
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1504 if self.mode in "aw":
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1505 self._loaded = True
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1506
57338 d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1507 if self.pax_headers:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1508 buf = self.tarinfo.create_pax_global_header(self.pax_headers.copy())
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1509 self.fileobj.write(buf)
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1510 self.offset += len(buf)
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1511 except:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1512 if not self._extfileobj:
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1513 self.fileobj.close()
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1514 self.closed = True
d8019d1d56f3 Merged revisions 76383 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1515 raise
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1516
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1517 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1518 # Below are the classmethods which act as alternate constructors to the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1519 # TarFile class. The open() method is the only one that is needed for
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1520 # public use; it is the "super"-constructor and is able to select an
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1521 # adequate "sub"-constructor for a particular compression using the mapping
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1522 # from OPEN_METH.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1523 #
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1524 # This concept allows one to subclass TarFile without losing the comfort of
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1525 # the super-constructor. A sub-constructor is registered and made available
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1526 # by adding it to the mapping in OPEN_METH.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1527
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1528 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1529 def open(cls, name=None, mode="r", fileobj=None, bufsize=RECORDSIZE, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1530 """Open a tar archive for reading, writing or appending. Return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1531 an appropriate TarFile class.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1532
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1533 mode:
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1534 'r' or 'r:*' open for reading with transparent compression
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1535 'r:' open for reading exclusively uncompressed
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1536 'r:gz' open for reading with gzip compression
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1537 'r:bz2' open for reading with bzip2 compression
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1538 'r:xz' open for reading with lzma compression
40893 32c4897b5d46 Merged revisions 53623-53858 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40775 diff changeset	1539 'a' or 'a:' open for appending, creating the file if necessary
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1540 'w' or 'w:' open for writing without compression
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1541 'w:gz' open for writing with gzip compression
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1542 'w:bz2' open for writing with bzip2 compression
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1543 'w:xz' open for writing with lzma compression
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1544
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1545 'r\|*' open a stream of tar blocks with transparent compression
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1546 'r\|' open an uncompressed stream of tar blocks for reading
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1547 'r\|gz' open a gzip compressed stream of tar blocks
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1548 'r\|bz2' open a bzip2 compressed stream of tar blocks
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1549 'r\|xz' open an lzma compressed stream of tar blocks
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1550 'w\|' open an uncompressed stream for writing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1551 'w\|gz' open a gzip compressed stream for writing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1552 'w\|bz2' open a bzip2 compressed stream for writing
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1553 'w\|xz' open an lzma compressed stream for writing
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1554 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1555
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1556 if not name and not fileobj:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1557 raise ValueError("nothing to open")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1558
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1559 if mode in ("r", "r:*"):
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1560 # Find out which *open() is appropriate for opening the file.
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1561 for comptype in cls.OPEN_METH:
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1562 func = getattr(cls, cls.OPEN_METH[comptype])
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1563 if fileobj is not None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1564 saved_pos = fileobj.tell()
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1565 try:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1566 return func(name, "r", fileobj, **kwargs)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1567 except (ReadError, CompressionError) as e:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1568 if fileobj is not None:
67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1569 fileobj.seek(saved_pos)
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1570 continue
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1571 raise ReadError("file could not be opened successfully")
34697 79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1572
79302e50910e Patch #918101: Add tarfile open mode r\|* for auto-detection of the Martin v. Löwis <martin@v.loewis.de> parents: 34688 diff changeset	1573 elif ":" in mode:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1574 filemode, comptype = mode.split(":", 1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1575 filemode = filemode or "r"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1576 comptype = comptype or "tar"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1577
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1578 # Select the *open() function according to
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1579 # given compression.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1580 if comptype in cls.OPEN_METH:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1581 func = getattr(cls, cls.OPEN_METH[comptype])
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1582 else:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1583 raise CompressionError("unknown compression type %r" % comptype)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1584 return func(name, filemode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1585
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1586 elif "\|" in mode:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1587 filemode, comptype = mode.split("\|", 1)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1588 filemode = filemode or "r"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1589 comptype = comptype or "tar"
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1590
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1591 if filemode not in "rw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1592 raise ValueError("mode must be 'r' or 'w'")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1593
64915 0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1594 stream = _Stream(name, filemode, comptype, fileobj, bufsize)
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1595 try:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1596 t = cls(name, filemode, stream, **kwargs)
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1597 except:
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1598 stream.close()
0c77d00f4f82 Further tarfile / test_tarfile cleanup Antoine Pitrou <solipsis@pitrou.net> parents: 64908 diff changeset	1599 raise
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1600 t._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1601 return t
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1602
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1603 elif mode in "aw":
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1604 return cls.taropen(name, mode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1605
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1606 raise ValueError("undiscernible mode")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1607
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1608 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1609 def taropen(cls, name, mode="r", fileobj=None, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1610 """Open uncompressed tar archive name for reading or writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1611 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1612 if len(mode) > 1 or mode not in "raw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1613 raise ValueError("mode must be 'r', 'a' or 'w'")
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1614 return cls(name, mode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1615
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1616 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1617 def gzopen(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1618 """Open gzip compressed tar archive name for reading or writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1619 Appending is not allowed.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1620 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1621 if len(mode) > 1 or mode not in "rw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1622 raise ValueError("mode must be 'r' or 'w'")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1623
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1624 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1625 import gzip
28576 1bd50c28203f Fix test_tarfile failure when gzip is not available Neal Norwitz <nnorwitz@gmail.com> parents: 28222 diff changeset	1626 gzip.GzipFile
1bd50c28203f Fix test_tarfile failure when gzip is not available Neal Norwitz <nnorwitz@gmail.com> parents: 28222 diff changeset	1627 except (ImportError, AttributeError):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1628 raise CompressionError("gzip module is not available")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1629
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1630 extfileobj = fileobj is not None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1631 try:
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1632 fileobj = gzip.GzipFile(name, mode + "b", compresslevel, fileobj)
64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1633 t = cls.taropen(name, mode, fileobj, **kwargs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1634 except IOError:
71855 843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1635 if not extfileobj and fileobj is not None:
64908 d560eece0857 Try to fix test_tarfile issues on Windows buildbots by closing file Antoine Pitrou <solipsis@pitrou.net> parents: 62898 diff changeset	1636 fileobj.close()
71855 843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1637 if fileobj is None:
843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1638 raise
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1639 raise ReadError("not a gzip file")
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1640 except:
71855 843cd43206b4 Fix #11513: wrong exception handling for the case that GzipFile itself raises an IOError. Georg Brandl <georg@python.org> parents: 71540 diff changeset	1641 if not extfileobj and fileobj is not None:
65742 64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1642 fileobj.close()
64ef61d9ffdc Issue #10233: Close file objects in a timely manner in the tarfile module Antoine Pitrou <solipsis@pitrou.net> parents: 65705 diff changeset	1643 raise
64908 d560eece0857 Try to fix test_tarfile issues on Windows buildbots by closing file Antoine Pitrou <solipsis@pitrou.net> parents: 62898 diff changeset	1644 t._extfileobj = extfileobj
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1645 return t
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1646
34421 3316aa540e68 Use decorators. Guido van Rossum <guido@python.org> parents: 34293 diff changeset	1647 @classmethod
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1648 def bz2open(cls, name, mode="r", fileobj=None, compresslevel=9, **kwargs):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1649 """Open bzip2 compressed tar archive name for reading or writing.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1650 Appending is not allowed.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1651 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1652 if len(mode) > 1 or mode not in "rw":
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1653 raise ValueError("mode must be 'r' or 'w'.")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1654
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1655 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1656 import bz2
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1657 except ImportError:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1658 raise CompressionError("bz2 module is not available")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1659
77353 544e64e18a8a Make BZ2File's fileobj support easier to use. Nadeem Vawda <nadeem.vawda@gmail.com> parents: 77015 diff changeset	1660 fileobj = bz2.BZ2File(fileobj or name, mode,
544e64e18a8a Make BZ2File's fileobj support easier to use. Nadeem Vawda <nadeem.vawda@gmail.com> parents: 77015 diff changeset	1661 compresslevel=compresslevel)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1662
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1663 try:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1664 t = cls.taropen(name, mode, fileobj, **kwargs)
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	1665 except (IOError, EOFError):
64908 d560eece0857 Try to fix test_tarfile issues on Windows buildbots by closing file Antoine Pitrou <solipsis@pitrou.net> parents: 62898 diff changeset	1666 fileobj.close()
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1667 raise ReadError("not a bzip2 file")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1668 t._extfileobj = False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1669 return t
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1670
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1671 @classmethod
74495 b86b54fcb5c2 Issue #5689: Avoid excessive memory usage by using the default lzma preset. Lars Gustäbel <lars@gustaebel.de> parents: 74278 diff changeset	1672 def xzopen(cls, name, mode="r", fileobj=None, preset=None, **kwargs):
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1673 """Open lzma compressed tar archive name for reading or writing.
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1674 Appending is not allowed.
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1675 """
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1676 if mode not in ("r", "w"):
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1677 raise ValueError("mode must be 'r' or 'w'")
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1678
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1679 try:
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1680 import lzma
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1681 except ImportError:
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1682 raise CompressionError("lzma module is not available")
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1683
77356 eae12226a66d Simplify usage of LZMAFile's fileobj support, like with BZ2File. Nadeem Vawda <nadeem.vawda@gmail.com> parents: 77353 diff changeset	1684 fileobj = lzma.LZMAFile(fileobj or name, mode, preset=preset)
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1685
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1686 try:
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1687 t = cls.taropen(name, mode, fileobj, **kwargs)
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1688 except (lzma.LZMAError, EOFError):
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1689 fileobj.close()
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1690 raise ReadError("not an lzma file")
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1691 t._extfileobj = False
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1692 return t
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1693
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1694 # All *open() methods are registered here.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1695 OPEN_METH = {
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1696 "tar": "taropen", # uncompressed tar
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1697 "gz": "gzopen", # gzip compressed tar
73927 899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1698 "bz2": "bz2open", # bzip2 compressed tar
899a8c7b2310 Issue #5689: Add support for lzma compression to the tarfile module. Lars Gustäbel <lars@gustaebel.de> parents: 73871 diff changeset	1699 "xz": "xzopen" # lzma compressed tar
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1700 }
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1701
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1702 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1703 # The public methods which TarFile provides:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1704
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1705 def close(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1706 """Close the TarFile. In write-mode, two finishing zero blocks are
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1707 appended to the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1708 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1709 if self.closed:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1710 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1711
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1712 if self.mode in "aw":
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1713 self.fileobj.write(NUL * (BLOCKSIZE * 2))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1714 self.offset += (BLOCKSIZE * 2)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1715 # fill up the end with zero-blocks
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1716 # (like option -b20 for tar does)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1717 blocks, remainder = divmod(self.offset, RECORDSIZE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1718 if remainder > 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1719 self.fileobj.write(NUL * (RECORDSIZE - remainder))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1720
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1721 if not self._extfileobj:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1722 self.fileobj.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1723 self.closed = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1724
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1725 def getmember(self, name):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1726 """Return a TarInfo object for member `name'. If `name' can not be
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1727 found in the archive, KeyError is raised. If a member occurs more
51776 d06a3ebf306f Merged revisions 69846 via svnmerge from Mark Dickinson <dickinsm@gmail.com> parents: 50132 diff changeset	1728 than once in the archive, its last occurrence is assumed to be the
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1729 most up-to-date version.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1730 """
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1731 tarinfo = self._getmember(name)
725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1732 if tarinfo is None:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	1733 raise KeyError("filename %r not found" % name)
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1734 return tarinfo
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1735
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1736 def getmembers(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1737 """Return the members of the archive as a list of TarInfo objects. The
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1738 list has the same order as the members in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1739 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1740 self._check()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1741 if not self._loaded: # if we want to obtain a list of
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1742 self._load() # all members, we first have to
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1743 # scan the whole archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1744 return self.members
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1745
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1746 def getnames(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1747 """Return the members of the archive as a list of their names. It has
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1748 the same order as the list returned by getmembers().
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1749 """
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1750 return [tarinfo.name for tarinfo in self.getmembers()]
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1751
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1752 def gettarinfo(self, name=None, arcname=None, fileobj=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1753 """Create a TarInfo object for either the file `name' or the file
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1754 object `fileobj' (using os.fstat on its file descriptor). You can
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1755 modify some of the TarInfo's attributes before you add it using
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1756 addfile(). If given, `arcname' specifies an alternative name for the
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1757 file in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1758 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1759 self._check("aw")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1760
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1761 # When fileobj is given, replace name by
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1762 # fileobj's real name.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1763 if fileobj is not None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1764 name = fileobj.name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1765
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1766 # Building the name of the member in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1767 # Backward slashes are converted to forward slashes,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1768 # Absolute paths are turned to relative paths.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1769 if arcname is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1770 arcname = name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1771 drv, arcname = os.path.splitdrive(arcname)
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1772 arcname = arcname.replace(os.sep, "/")
9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	1773 arcname = arcname.lstrip("/")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1774
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1775 # Now, fill the TarInfo object with
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1776 # information specific for the file.
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1777 tarinfo = self.tarinfo()
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1778 tarinfo.tarfile = self
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1779
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1780 # Use os.stat or os.lstat, depending on platform
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1781 # and if symlinks shall be resolved.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1782 if fileobj is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1783 if hasattr(os, "lstat") and not self.dereference:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1784 statres = os.lstat(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1785 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1786 statres = os.stat(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1787 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1788 statres = os.fstat(fileobj.fileno())
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1789 linkname = ""
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1790
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1791 stmd = statres.st_mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1792 if stat.S_ISREG(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1793 inode = (statres.st_ino, statres.st_dev)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1794 if not self.dereference and statres.st_nlink > 1 and \
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1795 inode in self.inodes and arcname != self.inodes[inode]:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1796 # Is it a hardlink to an already
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1797 # archived file?
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1798 type = LNKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1799 linkname = self.inodes[inode]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1800 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1801 # The inode is added only if its valid.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1802 # For win32 it is always 0.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1803 type = REGTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1804 if inode[0]:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1805 self.inodes[inode] = arcname
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1806 elif stat.S_ISDIR(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1807 type = DIRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1808 elif stat.S_ISFIFO(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1809 type = FIFOTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1810 elif stat.S_ISLNK(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1811 type = SYMTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1812 linkname = os.readlink(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1813 elif stat.S_ISCHR(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1814 type = CHRTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1815 elif stat.S_ISBLK(stmd):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1816 type = BLKTYPE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1817 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1818 return None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1819
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1820 # Fill the TarInfo object with all
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1821 # information we can get.
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1822 tarinfo.name = arcname
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1823 tarinfo.mode = stmd
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1824 tarinfo.uid = statres.st_uid
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1825 tarinfo.gid = statres.st_gid
62043 cea5ef682bdd Merged revisions 81665 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 61092 diff changeset	1826 if type == REGTYPE:
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1827 tarinfo.size = statres.st_size
33208 04864c39b006 Replace tricky and/or with straight-forward if:else: Martin v. Löwis <martin@v.loewis.de> parents: 33179 diff changeset	1828 else:
40644 1c26505c1de6 Rip out 'long' and 'L'-suffixed integer literals. Guido van Rossum <guido@python.org> parents: 40587 diff changeset	1829 tarinfo.size = 0
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1830 tarinfo.mtime = statres.st_mtime
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1831 tarinfo.type = type
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1832 tarinfo.linkname = linkname
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1833 if pwd:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1834 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1835 tarinfo.uname = pwd.getpwuid(tarinfo.uid)[0]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1836 except KeyError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1837 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1838 if grp:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1839 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1840 tarinfo.gname = grp.getgrgid(tarinfo.gid)[0]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1841 except KeyError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1842 pass
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1843
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1844 if type in (CHRTYPE, BLKTYPE):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1845 if hasattr(os, "major") and hasattr(os, "minor"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1846 tarinfo.devmajor = os.major(statres.st_rdev)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1847 tarinfo.devminor = os.minor(statres.st_rdev)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1848 return tarinfo
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1849
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1850 def list(self, verbose=True):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1851 """Print a table of contents to sys.stdout. If `verbose' is False, only
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1852 the names of the members are printed. If it is True, an `ls -l'-like
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1853 output is produced.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1854 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1855 self._check()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1856
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1857 for tarinfo in self:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1858 if verbose:
76949 492e6c6a01bb #14807: move undocumented tarfile.filemode() to stat.filemode(). Add tarfile.filemode alias with deprecation warning. Giampaolo Rodola' <g.rodola@gmail.com> parents: 76918 diff changeset	1859 print(stat.filemode(tarinfo.mode), end=' ')
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1860 print("%s/%s" % (tarinfo.uname or tarinfo.uid,
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1861 tarinfo.gname or tarinfo.gid), end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1862 if tarinfo.ischr() or tarinfo.isblk():
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1863 print("%10s" % ("%d,%d" \
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1864 % (tarinfo.devmajor, tarinfo.devminor)), end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1865 else:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1866 print("%10d" % tarinfo.size, end=' ')
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1867 print("%d-%02d-%02d %02d:%02d:%02d" \
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1868 % time.localtime(tarinfo.mtime)[:6], end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1869
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	1870 print(tarinfo.name + ("/" if tarinfo.isdir() else ""), end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1871
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1872 if verbose:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1873 if tarinfo.issym():
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1874 print("->", tarinfo.linkname, end=' ')
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1875 if tarinfo.islnk():
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1876 print("link to", tarinfo.linkname, end=' ')
4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	1877 print()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1878
67771 1f215fda2383 Issue #11014: Make 'filter' argument in tarfile.Tarfile.add() into a Raymond Hettinger <python@rcn.com> parents: 65879 diff changeset	1879 def add(self, name, arcname=None, recursive=True, exclude=None, *, filter=None):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1880 """Add the file `name' to the archive. `name' may be any type of file
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1881 (directory, fifo, symbolic link, etc.). If given, `arcname'
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1882 specifies an alternative name for the file in the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1883 Directories are added recursively by default. This can be avoided by
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 42084 diff changeset	1884 setting `recursive' to False. `exclude' is a function that should
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1885 return True for each filename to be excluded. `filter' is a function
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1886 that expects a TarInfo object argument and returns the changed
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1887 TarInfo object, if it returns None the TarInfo object will be
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1888 excluded from the archive.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1889 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1890 self._check("aw")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1891
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1892 if arcname is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1893 arcname = name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1894
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 42084 diff changeset	1895 # Exclude pathnames.
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1896 if exclude is not None:
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1897 import warnings
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1898 warnings.warn("use the filter argument instead",
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1899 DeprecationWarning, 2)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1900 if exclude(name):
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1901 self._dbg(2, "tarfile: Excluded %r" % name)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1902 return
42132 863ed7cd45dc Merged revisions 56020-56124 via svnmerge from Guido van Rossum <guido@python.org> parents: 42084 diff changeset	1903
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1904 # Skip if somebody tries to archive the archive...
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	1905 if self.name is not None and os.path.abspath(name) == self.name:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1906 self._dbg(2, "tarfile: Skipped %r" % name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1907 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1908
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1909 self._dbg(1, name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1910
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1911 # Create a TarInfo object from the file.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1912 tarinfo = self.gettarinfo(name, arcname)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1913
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1914 if tarinfo is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1915 self._dbg(1, "tarfile: Unsupported type %r" % name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1916 return
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1917
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1918 # Change or exclude the TarInfo object.
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1919 if filter is not None:
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1920 tarinfo = filter(tarinfo)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1921 if tarinfo is None:
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1922 self._dbg(2, "tarfile: Excluded %r" % name)
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1923 return
51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1924
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1925 # Append the tar header and data to the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1926 if tarinfo.isreg():
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1927 f = bltn_open(name, "rb")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1928 self.addfile(tarinfo, f)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1929 f.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1930
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1931 elif tarinfo.isdir():
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1932 self.addfile(tarinfo)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1933 if recursive:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1934 for f in os.listdir(name):
55917 51baad28fe7a Merged revisions 74750 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 55748 diff changeset	1935 self.add(os.path.join(name, f), os.path.join(arcname, f),
67771 1f215fda2383 Issue #11014: Make 'filter' argument in tarfile.Tarfile.add() into a Raymond Hettinger <python@rcn.com> parents: 65879 diff changeset	1936 recursive, exclude, filter=filter)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1937
35466 85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1938 else:
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1939 self.addfile(tarinfo)
85960ca4fc08 Patch #1168594: set sizes of non-regular files to zero. Fixes #1167128. Martin v. Löwis <martin@v.loewis.de> parents: 35403 diff changeset	1940
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1941 def addfile(self, tarinfo, fileobj=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1942 """Add the TarInfo object `tarinfo' to the archive. If `fileobj' is
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1943 given, tarinfo.size bytes are read from it and added to the archive.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1944 You can create TarInfo objects using gettarinfo().
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1945 On Windows platforms, `fileobj' should always be opened with mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1946 'rb' to avoid irritation about the file size.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1947 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1948 self._check("aw")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1949
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1950 tarinfo = copy.copy(tarinfo)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1951
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	1952 buf = tarinfo.tobuf(self.format, self.encoding, self.errors)
40450 cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1953 self.fileobj.write(buf)
cc992d75d5b3 Four months of trunk changes (including a few releases...) Thomas Wouters <thomas@python.org> parents: 39649 diff changeset	1954 self.offset += len(buf)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1955
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1956 # If there's data to follow, append it.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1957 if fileobj is not None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1958 copyfileobj(fileobj, self.fileobj, tarinfo.size)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1959 blocks, remainder = divmod(tarinfo.size, BLOCKSIZE)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1960 if remainder > 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1961 self.fileobj.write(NUL * (BLOCKSIZE - remainder))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1962 blocks += 1
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1963 self.offset += blocks * BLOCKSIZE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1964
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	1965 self.members.append(tarinfo)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	1966
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1967 def extractall(self, path=".", members=None):
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1968 """Extract all members from the archive to the current working
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1969 directory and set owner, modification time and permissions on
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1970 directories afterwards. `path' specifies a different directory
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1971 to extract to. `members' is optional and must be a subset of the
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1972 list returned by getmembers().
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1973 """
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1974 directories = []
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1975
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1976 if members is None:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1977 members = self
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1978
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1979 for tarinfo in members:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1980 if tarinfo.isdir():
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	1981 # Extract directories with a safe mode.
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1982 directories.append(tarinfo)
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	1983 tarinfo = copy.copy(tarinfo)
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	1984 tarinfo.mode = 0o700
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	1985 # Do not set_attrs directories, as we will do that further down
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	1986 self.extract(tarinfo, path, set_attrs=not tarinfo.isdir())
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1987
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1988 # Reverse sort directories.
45049 90e86ee3684b Convert some custom sort comparison functions to equivalent key functions. Raymond Hettinger <python@rcn.com> parents: 44535 diff changeset	1989 directories.sort(key=lambda a: a.name)
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1990 directories.reverse()
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1991
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1992 # Set correct owner, mtime and filemode on directories.
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1993 for tarinfo in directories:
44535 420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	1994 dirpath = os.path.join(path, tarinfo.name)
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	1995 try:
44535 420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	1996 self.chown(tarinfo, dirpath)
420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	1997 self.utime(tarinfo, dirpath)
420fe6d7d5f4 Merged revisions 59703-59773 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 44104 diff changeset	1998 self.chmod(tarinfo, dirpath)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	1999 except ExtractError as e:
34688 1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2000 if self.errorlevel > 1:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2001 raise
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2002 else:
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2003 self._dbg(1, "tarfile: %s" % e)
1ac24e9183e5 Patch #1043890: tarfile: add extractall() method. Martin v. Löwis <martin@v.loewis.de> parents: 34680 diff changeset	2004
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2005 def extract(self, member, path="", set_attrs=True):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2006 """Extract a member from the archive to the current working directory,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2007 using its full name. Its file information is extracted as accurately
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2008 as possible. `member' may be a filename or a TarInfo object. You can
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2009 specify a different directory using `path'. File attributes (owner,
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2010 mtime, mode) are set unless `set_attrs' is False.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2011 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2012 self._check("r")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2013
43577 ace45b361e87 Patch# 1258 by Christian Heimes: kill basestring. Guido van Rossum <guido@python.org> parents: 43020 diff changeset	2014 if isinstance(member, str):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2015 tarinfo = self.getmember(member)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2016 else:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2017 tarinfo = member
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2018
32769 983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2019 # Prepare the link target for makelink().
983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2020 if tarinfo.islnk():
983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2021 tarinfo._link_target = os.path.join(path, tarinfo.linkname)
983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2022
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2023 try:
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2024 self._extract_member(tarinfo, os.path.join(path, tarinfo.name),
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2025 set_attrs=set_attrs)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2026 except EnvironmentError as e:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2027 if self.errorlevel > 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2028 raise
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2029 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2030 if e.filename is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2031 self._dbg(1, "tarfile: %s" % e.strerror)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2032 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2033 self._dbg(1, "tarfile: %s %r" % (e.strerror, e.filename))
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2034 except ExtractError as e:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2035 if self.errorlevel > 1:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2036 raise
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2037 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2038 self._dbg(1, "tarfile: %s" % e)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2039
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2040 def extractfile(self, member):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2041 """Extract a member from the archive as a file object. `member' may be
76770 254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	2042 a filename or a TarInfo object. If `member' is a regular file or a
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	2043 link, an io.BufferedReader object is returned. Otherwise, None is
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	2044 returned.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2045 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2046 self._check("r")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2047
43577 ace45b361e87 Patch# 1258 by Christian Heimes: kill basestring. Guido van Rossum <guido@python.org> parents: 43020 diff changeset	2048 if isinstance(member, str):
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2049 tarinfo = self.getmember(member)
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2050 else:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2051 tarinfo = member
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2052
76770 254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	2053 if tarinfo.isreg() or tarinfo.type not in SUPPORTED_TYPES:
254cb4f5d0ff Issue #13815: TarFile.extractfile() now returns io.BufferedReader objects. Lars Gustäbel <lars@gustaebel.de> parents: 76539 diff changeset	2054 # Members with unknown types are treated as regular files.
76918 ab6496b98ac4 Issue #13815: Resurrect the ExFileObject class. Lars Gustäbel <lars@gustaebel.de> parents: 76770 diff changeset	2055 return self.fileobject(self, tarinfo)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2056
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2057 elif tarinfo.islnk() or tarinfo.issym():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2058 if isinstance(self.fileobj, _Stream):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2059 # A small but ugly workaround for the case that someone tries
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2060 # to extract a (sym)link as a file-object from a non-seekable
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2061 # stream of tar blocks.
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2062 raise StreamError("cannot extract (sym)link as file object")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2063 else:
35276 4adbd331d335 Fix all wrong instances of "it's". Georg Brandl <georg@python.org> parents: 34818 diff changeset	2064 # A (sym)link's file object is its target's file object.
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2065 return self.extractfile(self._find_link_target(tarinfo))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2066 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2067 # If there's no data associated with the member (directory, chrdev,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2068 # blkdev, etc.), return None instead of a file object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2069 return None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2070
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2071 def _extract_member(self, tarinfo, targetpath, set_attrs=True):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2072 """Extract the TarInfo object tarinfo to a physical
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2073 file called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2074 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2075 # Fetch the TarInfo object for the given name
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2076 # and build the destination pathname, replacing
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2077 # forward slashes to platform specific separators.
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	2078 targetpath = targetpath.rstrip("/")
9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	2079 targetpath = targetpath.replace("/", os.sep)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2080
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2081 # Create all upper directories.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2082 upperdirs = os.path.dirname(targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2083 if upperdirs and not os.path.exists(upperdirs):
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2084 # Create directories that are not part of the archive with
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2085 # default permissions.
40737 6ebad2a6b990 Merged revisions 53451-53537 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40644 diff changeset	2086 os.makedirs(upperdirs)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2087
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2088 if tarinfo.islnk() or tarinfo.issym():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2089 self._dbg(1, "%s -> %s" % (tarinfo.name, tarinfo.linkname))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2090 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2091 self._dbg(1, tarinfo.name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2092
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2093 if tarinfo.isreg():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2094 self.makefile(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2095 elif tarinfo.isdir():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2096 self.makedir(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2097 elif tarinfo.isfifo():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2098 self.makefifo(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2099 elif tarinfo.ischr() or tarinfo.isblk():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2100 self.makedev(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2101 elif tarinfo.islnk() or tarinfo.issym():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2102 self.makelink(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2103 elif tarinfo.type not in SUPPORTED_TYPES:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2104 self.makeunknown(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2105 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2106 self.makefile(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2107
65879 be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2108 if set_attrs:
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2109 self.chown(tarinfo, targetpath)
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2110 if not tarinfo.issym():
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2111 self.chmod(tarinfo, targetpath)
be1d7f619fc0 Issue #10184: Touch directories only once when extracting a tarfile. Martin v. Löwis <martin@v.loewis.de> parents: 65742 diff changeset	2112 self.utime(tarinfo, targetpath)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2113
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2114 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2115 # Below are the different file methods. They are called via
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2116 # _extract_member() when extract() is called. They can be replaced in a
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2117 # subclass to implement other functionality.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2118
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2119 def makedir(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2120 """Make a directory called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2121 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2122 try:
45192 142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2123 # Use a safe mode for the directory, the real mode is set
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2124 # later in _extract_member().
142a11b3057e Merged revisions 60481,60485,60489-60492,60494-60496,60498-60499,60501-60503,60505-60506,60508-60509,60523-60524,60532,60543,60545,60547-60548,60552,60554,60556-60559,60561-60562,60568-60598,60600-60616 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 45049 diff changeset	2125 os.mkdir(targetpath, 0o700)
73169 e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 72924 diff changeset	2126 except FileExistsError:
e4d44c2e8e81 Simplify and remove few dependencies on 'errno', thanks to PEP 3151. Florent Xicluna <florent.xicluna@gmail.com> parents: 72924 diff changeset	2127 pass
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2128
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2129 def makefile(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2130 """Make a file called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2131 """
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2132 source = self.fileobj
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2133 source.seek(tarinfo.offset_data)
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	2134 target = bltn_open(targetpath, "wb")
65705 050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2135 if tarinfo.sparse is not None:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2136 for offset, size in tarinfo.sparse:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2137 target.seek(offset)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2138 copyfileobj(source, target, size)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2139 else:
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2140 copyfileobj(source, target, tarinfo.size)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2141 target.seek(tarinfo.size)
050f0f7be11e Add read support for all missing variants of the GNU sparse Lars Gustäbel <lars@gustaebel.de> parents: 65084 diff changeset	2142 target.truncate()
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2143 target.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2144
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2145 def makeunknown(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2146 """Make a file from a TarInfo object with an unknown type
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2147 at targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2148 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2149 self.makefile(tarinfo, targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2150 self._dbg(1, "tarfile: Unknown file type %r, " \
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2151 "extracted as regular file." % tarinfo.type)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2152
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2153 def makefifo(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2154 """Make a fifo called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2155 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2156 if hasattr(os, "mkfifo"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2157 os.mkfifo(targetpath)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2158 else:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2159 raise ExtractError("fifo not supported by system")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2160
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2161 def makedev(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2162 """Make a character or block device called targetpath.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2163 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2164 if not hasattr(os, "mknod") or not hasattr(os, "makedev"):
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2165 raise ExtractError("special devices not supported by system")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2166
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2167 mode = tarinfo.mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2168 if tarinfo.isblk():
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2169 mode \|= stat.S_IFBLK
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2170 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2171 mode \|= stat.S_IFCHR
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2172
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2173 os.mknod(targetpath, mode,
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2174 os.makedev(tarinfo.devmajor, tarinfo.devminor))
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2175
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2176 def makelink(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2177 """Make a (symbolic) link called targetpath. If it cannot be created
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2178 (platform limitation), we try to make a copy of the referenced file
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2179 instead of a link.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2180 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2181 try:
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2182 # For systems that support symbolic and hard links.
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2183 if tarinfo.issym():
55748 9a78e3d90617 Merged revisions 74571 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 52269 diff changeset	2184 os.symlink(tarinfo.linkname, targetpath)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2185 else:
32769 983a91b50f6e SF #857297 and 916874, improve handling of hard links when extracting Neal Norwitz <nnorwitz@gmail.com> parents: 32768 diff changeset	2186 # See extract().
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2187 if os.path.exists(tarinfo._link_target):
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2188 os.link(tarinfo._link_target, targetpath)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2189 else:
64929 260dc36c2c86 Fix a line that got hacked up by r82659. Brian Curtin <brian.curtin@gmail.com> parents: 64915 diff changeset	2190 self._extract_member(self._find_link_target(tarinfo),
260dc36c2c86 Fix a line that got hacked up by r82659. Brian Curtin <brian.curtin@gmail.com> parents: 64915 diff changeset	2191 targetpath)
62898 5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	2192 except symlink_exception:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2193 try:
62898 5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	2194 self._extract_member(self._find_link_target(tarinfo),
5ae7df6c93ed Fix the breakage of Lib/tarfile.py on non-Windows platforms due to Brian Curtin <brian.curtin@gmail.com> parents: 62874 diff changeset	2195 targetpath)
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2196 except KeyError:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2197 raise ExtractError("unable to resolve link inside archive")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2198
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2199 def chown(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2200 """Set owner of targetpath according to tarinfo.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2201 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2202 if pwd and hasattr(os, "geteuid") and os.geteuid() == 0:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2203 # We have to be root to do so.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2204 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2205 g = grp.getgrnam(tarinfo.gname)[2]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2206 except KeyError:
72272 2bc122347351 Issue #12841: Fix tarfile extraction of non-existent uids/gids. Lars Gustäbel <lars@gustaebel.de> parents: 71855 diff changeset	2207 g = tarinfo.gid
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2208 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2209 u = pwd.getpwnam(tarinfo.uname)[2]
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2210 except KeyError:
72272 2bc122347351 Issue #12841: Fix tarfile extraction of non-existent uids/gids. Lars Gustäbel <lars@gustaebel.de> parents: 71855 diff changeset	2211 u = tarinfo.uid
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2212 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2213 if tarinfo.issym() and hasattr(os, "lchown"):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2214 os.lchown(targetpath, u, g)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2215 else:
79480 15949f34e08f #16135: Removal of OS/2 support (Python code partial cleanup) Jesus Cea <jcea@jcea.es> parents: 77356 diff changeset	2216 os.chown(targetpath, u, g)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2217 except EnvironmentError as e:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2218 raise ExtractError("could not change owner")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2219
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2220 def chmod(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2221 """Set file permissions of targetpath according to tarinfo.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2222 """
28214 d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2223 if hasattr(os, 'chmod'):
d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2224 try:
d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2225 os.chmod(targetpath, tarinfo.mode)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2226 except EnvironmentError as e:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2227 raise ExtractError("could not change mode")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2228
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2229 def utime(self, tarinfo, targetpath):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2230 """Set modification time of targetpath according to tarinfo.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2231 """
28214 d4b262fbd04a Test that os.utime and os.chmod actually exist before using them. Jack Jansen <jack.jansen@cwi.nl> parents: 27972 diff changeset	2232 if not hasattr(os, 'utime'):
28222 eca697e06a07 Somebody must not have run the test before checking this in -- it had Tim Peters <tim.peters@gmail.com> parents: 28217 diff changeset	2233 return
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2234 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2235 os.utime(targetpath, (tarinfo.mtime, tarinfo.mtime))
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 40582 diff changeset	2236 except EnvironmentError as e:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2237 raise ExtractError("could not change modification time")
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2238
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2239 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2240 def next(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2241 """Return the next member of the archive as a TarInfo object, when
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2242 TarFile is opened for reading. Return None if there is no more
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2243 available.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2244 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2245 self._check("ra")
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2246 if self.firstmember is not None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2247 m = self.firstmember
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2248 self.firstmember = None
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2249 return m
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2250
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2251 # Read the next block.
32604 08d90460ab4d [Patch 988444] Andrew M. Kuchling <amk@amk.ca> parents: 31314 diff changeset	2252 self.fileobj.seek(self.offset)
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2253 tarinfo = None
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2254 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2255 try:
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2256 tarinfo = self.tarinfo.fromtarfile(self)
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2257 except EOFHeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2258 if self.ignore_zeros:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2259 self._dbg(2, "0x%X: %s" % (self.offset, e))
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2260 self.offset += BLOCKSIZE
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2261 continue
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2262 except InvalidHeaderError as e:
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2263 if self.ignore_zeros:
40582 67287ca4e1e8 Merged revisions 53005-53303 via svnmerge from Thomas Wouters <thomas@python.org> parents: 40450 diff changeset	2264 self._dbg(2, "0x%X: %s" % (self.offset, e))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2265 self.offset += BLOCKSIZE
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2266 continue
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2267 elif self.offset == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2268 raise ReadError(str(e))
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2269 except EmptyHeaderError:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2270 if self.offset == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2271 raise ReadError("empty file")
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2272 except TruncatedHeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2273 if self.offset == 0:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2274 raise ReadError(str(e))
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2275 except SubsequentHeaderError as e:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2276 raise ReadError(str(e))
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2277 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2278
57394 d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2279 if tarinfo is not None:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2280 self.members.append(tarinfo)
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2281 else:
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2282 self._loaded = True
d6f8552fce79 Merged revisions 76443 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57334 diff changeset	2283
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2284 return tarinfo
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2285
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2286 #--------------------------------------------------------------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2287 # Little helper methods:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2288
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2289 def _getmember(self, name, tarinfo=None, normalize=False):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2290 """Find an archive member by name from bottom to top.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2291 If tarinfo is given, it is used as the starting point.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2292 """
33507 725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	2293 # Ensure that all members have been loaded.
725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	2294 members = self.getmembers()
725c44421a42 Patch #1029061: Always extract member names from the tarinfo. Martin v. Löwis <martin@v.loewis.de> parents: 33263 diff changeset	2295
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2296 # Limit the member search list up to tarinfo.
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2297 if tarinfo is not None:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2298 members = members[:members.index(tarinfo)]
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2299
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2300 if normalize:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2301 name = os.path.normpath(name)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2302
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2303 for member in reversed(members):
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2304 if normalize:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2305 member_name = os.path.normpath(member.name)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2306 else:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2307 member_name = member.name
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2308
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2309 if name == member_name:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2310 return member
32604 08d90460ab4d [Patch 988444] Andrew M. Kuchling <amk@amk.ca> parents: 31314 diff changeset	2311
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2312 def _load(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2313 """Read through the entire archive file and look for readable
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2314 members.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2315 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2316 while True:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2317 tarinfo = self.next()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2318 if tarinfo is None:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2319 break
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2320 self._loaded = True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2321
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2322 def _check(self, mode=None):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2323 """Check if TarFile is still open, and if the operation's mode
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2324 corresponds to TarFile's mode.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2325 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2326 if self.closed:
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 35804 diff changeset	2327 raise IOError("%s is closed" % self.__class__.__name__)
41554 06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2328 if mode is not None and self.mode not in mode:
06145fbc7ab9 Merged revisions 53952-54987 via svnmerge from Guido van Rossum <guido@python.org> parents: 41501 diff changeset	2329 raise IOError("bad operation for mode %r" % self.mode)
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2330
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2331 def _find_link_target(self, tarinfo):
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2332 """Find the target member of a symlink or hardlink member in the
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2333 archive.
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2334 """
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2335 if tarinfo.issym():
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2336 # Always search the entire archive.
76538 0adf4fd8df83 Issue #14160: TarFile.extractfile() failed to resolve symbolic links Lars Gustäbel <lars@gustaebel.de> parents: 74277 diff changeset	2337 linkname = "/".join(filter(None, (os.path.dirname(tarinfo.name), tarinfo.linkname)))
62047 bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2338 limit = None
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2339 else:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2340 # Search the archive before the link, because a hard link is
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2341 # just a reference to an already archived file.
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2342 linkname = tarinfo.linkname
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2343 limit = tarinfo
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2344
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2345 member = self._getmember(linkname, tarinfo=limit, normalize=True)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2346 if member is None:
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2347 raise KeyError("linkname %r not found" % linkname)
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2348 return member
bfbf2d650f26 Merged revisions 81667 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 62042 diff changeset	2349
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2350 def __iter__(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2351 """Provide an iterator object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2352 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2353 if self._loaded:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2354 return iter(self.members)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2355 else:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2356 return TarIter(self)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2357
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2358 def _dbg(self, level, msg):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2359 """Write debugging output to sys.stderr.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2360 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2361 if level <= self.debug:
40775 4cdf21a0dbbc Fix most trivially-findable print statements. Guido van Rossum <guido@python.org> parents: 40737 diff changeset	2362 print(msg, file=sys.stderr)
59304 c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2363
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2364 def __enter__(self):
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2365 self._check()
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2366 return self
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2367
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2368 def __exit__(self, type, value, traceback):
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2369 if type is None:
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2370 self.close()
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2371 else:
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2372 # An exception occurred. We must not call close() because
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2373 # it would try to write end-of-archive blocks and padding.
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2374 if not self._extfileobj:
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2375 self.fileobj.close()
c4fca778b573 Merged revisions 78623 via svnmerge from Lars Gustäbel <lars@gustaebel.de> parents: 57734 diff changeset	2376 self.closed = True
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2377 # class TarFile
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2378
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2379 class TarIter:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2380 """Iterator Class.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2381
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2382 for tarinfo in TarFile(...):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2383 suite...
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2384 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2385
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2386 def __init__(self, tarfile):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2387 """Construct a TarIter object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2388 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2389 self.tarfile = tarfile
34680 481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2390 self.index = 0
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2391 def __iter__(self):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2392 """Return iterator object.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2393 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2394 return self
41501 0c5019afc55b PEP 3114: rename .next() to .__next__() and add next() builtin. Georg Brandl <georg@python.org> parents: 40893 diff changeset	2395 def __next__(self):
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2396 """Return the next item using TarFile's next() method.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2397 When all members have been read, set TarFile as _loaded.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2398 """
34680 481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2399 # Fix for SF #1100429: Under rare circumstances it can
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2400 # happen that getmembers() is called during iteration,
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2401 # which will cause TarIter to stop prematurely.
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2402 if not self.tarfile._loaded:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2403 tarinfo = self.tarfile.next()
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2404 if not tarinfo:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2405 self.tarfile._loaded = True
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2406 raise StopIteration
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2407 else:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2408 try:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2409 tarinfo = self.tarfile.members[self.index]
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2410 except IndexError:
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2411 raise StopIteration
481397683eac Patch #1103407: Properly deal with tarfile iterators when untarring Martin v. Löwis <martin@v.loewis.de> parents: 34649 diff changeset	2412 self.index += 1
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2413 return tarinfo
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2414
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2415 #--------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2416 # exported functions
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2417 #--------------------
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2418 def is_tarfile(name):
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2419 """Return True if name points to a tar archive that we
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2420 are able to handle, else return False.
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2421 """
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2422 try:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2423 t = open(name)
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2424 t.close()
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2425 return True
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2426 except TarError:
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2427 return False
f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2428
41969 9d02a4adf574 Merged revisions 55631-55794 via svnmerge from Guido van Rossum <guido@python.org> parents: 41760 diff changeset	2429 bltn_open = open
27098 f9a5ed092660 SF #651082, tarfile module implementation from Lars Gustäbel Neal Norwitz <nnorwitz@gmail.com> parents: diff changeset	2430 open = TarFile.open

Mercurial > cpython-lru_cache-weakref

annotate Lib/tarfile.py @ 79691:05c8928362ee