cpython-lru_cache-weakref: Tools/scripts/ftpmirror.py annotate

annotate Tools/scripts/ftpmirror.py @ 87690:66c1c9f32567 default tip

Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859

author	Radomir Dopieralski <devel@sheep.art.pl>
date	Tue, 03 Dec 2013 16:02:48 +0100
parents	6cfe2982de42
children

rev	line source
59522 fcfe324663a6 convert shebang lines: python -> python3 Benjamin Peterson <benjamin@python.org> parents: 47100 diff changeset	1 #! /usr/bin/env python3
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	2
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	3 """Mirror a remote ftp subtree into a local directory tree.
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	4
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	5 usage: ftpmirror [-v] [-q] [-i] [-m] [-n] [-r] [-s pat]
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	6 [-l username [-p passwd [-a account]]]
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	7 hostname[:port] [remotedir [localdir]]
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	8 -v: verbose
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	9 -q: quiet
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	10 -i: interactive mode
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	11 -m: macintosh server (NCSA telnet 2.4) (implies -n -s '*.o')
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	12 -n: don't log in
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	13 -r: remove local files/directories no longer pertinent
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	14 -l username [-p passwd [-a account]]: login info (default .netrc or anonymous)
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	15 -s pat: skip files matching pattern
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	16 hostname: remote host w/ optional port separated by ':'
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	17 remotedir: remote directory (default initial)
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	18 localdir: local directory (default current)
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	19 """
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	20
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	21 import os
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	22 import sys
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	23 import time
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	24 import getopt
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	25 import ftplib
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	26 import netrc
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	27 from fnmatch import fnmatch
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	28
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	29 # Print usage message and exit
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	30 def usage(*args):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	31 sys.stdout = sys.stderr
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	32 for msg in args: print(msg)
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	33 print(__doc__)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	34 sys.exit(2)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	35
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	36 verbose = 1 # 0 for -q, 2 for -v
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	37 interactive = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	38 mac = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	39 rmok = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	40 nologin = 0
3374 0c8821bf6db5 don't die on unreadable files etc. Guido van Rossum <guido@python.org> parents: 2761 diff changeset	41 skippats = ['.', '..', '.mirrorinfo']
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	42
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	43 # Main program: parse command line and start processing
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	44 def main():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	45 global verbose, interactive, mac, rmok, nologin
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	46 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	47 opts, args = getopt.getopt(sys.argv[1:], 'a:bil:mnp:qrs:v')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	48 except getopt.error as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	49 usage(msg)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	50 login = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	51 passwd = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	52 account = ''
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	53 if not args: usage('hostname missing')
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	54 host = args[0]
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	55 port = 0
61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	56 if ':' in host:
61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	57 host, port = host.split(':', 1)
26536 6ea786528395 * when given, port should be converted to int Skip Montanaro <skip@pobox.com> parents: 26535 diff changeset	58 port = int(port)
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	59 try:
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	60 auth = netrc.netrc().authenticators(host)
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	61 if auth is not None:
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	62 login, account, passwd = auth
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	63 except (netrc.NetrcParseError, IOError):
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	64 pass
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	65 for o, a in opts:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	66 if o == '-l': login = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	67 if o == '-p': passwd = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	68 if o == '-a': account = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	69 if o == '-v': verbose = verbose + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	70 if o == '-q': verbose = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	71 if o == '-i': interactive = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	72 if o == '-m': mac = 1; nologin = 1; skippats.append('*.o')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	73 if o == '-n': nologin = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	74 if o == '-r': rmok = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	75 if o == '-s': skippats.append(a)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	76 remotedir = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	77 localdir = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	78 if args[1:]:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	79 remotedir = args[1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	80 if args[2:]:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	81 localdir = args[2]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	82 if args[3:]: usage('too many arguments')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	83 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	84 f = ftplib.FTP()
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	85 if verbose: print("Connecting to '%s%s'..." % (host,
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	86 (port and ":%d"%port or "")))
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	87 f.connect(host,port)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	88 if not nologin:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	89 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	90 print('Logging in as %r...' % (login or 'anonymous'))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	91 f.login(login, passwd, account)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	92 if verbose: print('OK.')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	93 pwd = f.pwd()
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	94 if verbose > 1: print('PWD =', repr(pwd))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	95 if remotedir:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	96 if verbose > 1: print('cwd(%s)' % repr(remotedir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	97 f.cwd(remotedir)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	98 if verbose > 1: print('OK.')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	99 pwd = f.pwd()
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	100 if verbose > 1: print('PWD =', repr(pwd))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	101 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	102 mirrorsubdir(f, localdir)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	103
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	104 # Core logic: mirror one subdirectory (recursively)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	105 def mirrorsubdir(f, localdir):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	106 pwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	107 if localdir and not os.path.isdir(localdir):
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	108 if verbose: print('Creating local directory', repr(localdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	109 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	110 makedir(localdir)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	111 except OSError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	112 print("Failed to establish local directory", repr(localdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	113 return
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	114 infofilename = os.path.join(localdir, '.mirrorinfo')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	115 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	116 text = open(infofilename, 'r').read()
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	117 except IOError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	118 text = '{}'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	119 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	120 info = eval(text)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	121 except (SyntaxError, NameError):
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	122 print('Bad mirror info in', repr(infofilename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	123 info = {}
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	124 subdirs = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	125 listing = []
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	126 if verbose: print('Listing remote directory %r...' % (pwd,))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	127 f.retrlines('LIST', listing.append)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	128 filesfound = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	129 for line in listing:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	130 if verbose > 1: print('-->', repr(line))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	131 if mac:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	132 # Mac listing has just filenames;
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	133 # trailing / means subdirectory
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	134 filename = line.strip()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	135 mode = '-'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	136 if filename[-1:] == '/':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	137 filename = filename[:-1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	138 mode = 'd'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	139 infostuff = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	140 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	141 # Parse, assuming a UNIX listing
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	142 words = line.split(None, 8)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	143 if len(words) < 6:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	144 if verbose > 1: print('Skipping short line')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	145 continue
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	146 filename = words[-1].lstrip()
8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	147 i = filename.find(" -> ")
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	148 if i >= 0:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	149 # words[0] had better start with 'l'...
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	150 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	151 print('Found symbolic link %r' % (filename,))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	152 linkto = filename[i+4:]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	153 filename = filename[:i]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	154 infostuff = words[-5:-1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	155 mode = words[0]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	156 skip = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	157 for pat in skippats:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	158 if fnmatch(filename, pat):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	159 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	160 print('Skip pattern', repr(pat), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	161 print('matches', repr(filename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	162 skip = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	163 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	164 if skip:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	165 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	166 if mode[0] == 'd':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	167 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	168 print('Remembering subdirectory', repr(filename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	169 subdirs.append(filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	170 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	171 filesfound.append(filename)
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	172 if filename in info and info[filename] == infostuff:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	173 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	174 print('Already have this version of',repr(filename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	175 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	176 fullname = os.path.join(localdir, filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	177 tempname = os.path.join(localdir, '@'+filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	178 if interactive:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	179 doit = askabout('file', filename, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	180 if not doit:
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	181 if filename not in info:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	182 info[filename] = 'Not retrieved'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	183 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	184 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	185 os.unlink(tempname)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	186 except OSError:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	187 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	188 if mode[0] == 'l':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	189 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	190 print("Creating symlink %r -> %r" % (filename, linkto))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	191 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	192 os.symlink(linkto, tempname)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	193 except IOError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	194 print("Can't create %r: %s" % (tempname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	195 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	196 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	197 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	198 fp = open(tempname, 'wb')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	199 except IOError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	200 print("Can't create %r: %s" % (tempname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	201 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	202 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	203 print('Retrieving %r from %r as %r...' % (filename, pwd, fullname))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	204 if verbose:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	205 fp1 = LoggingFile(fp, 1024, sys.stdout)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	206 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	207 fp1 = fp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	208 t0 = time.time()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	209 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	210 f.retrbinary('RETR ' + filename,
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	211 fp1.write, 8*1024)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	212 except ftplib.error_perm as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	213 print(msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	214 t1 = time.time()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	215 bytes = fp.tell()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	216 fp.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	217 if fp1 != fp:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	218 fp1.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	219 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	220 os.unlink(fullname)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	221 except OSError:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	222 pass # Ignore the error
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	223 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	224 os.rename(tempname, fullname)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	225 except OSError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	226 print("Can't rename %r to %r: %s" % (tempname, fullname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	227 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	228 info[filename] = infostuff
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	229 writedict(info, infofilename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	230 if verbose and mode[0] != 'l':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	231 dt = t1 - t0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	232 kbytes = bytes / 1024.0
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	233 print(int(round(kbytes)), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	234 print('Kbytes in', end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	235 print(int(round(dt)), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	236 print('seconds', end=' ')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	237 if t1 > t0:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	238 print('(~%d Kbytes/sec)' % \
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	239 int(round(kbytes/dt),))
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	240 print()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	241 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	242 # Remove files from info that are no longer remote
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	243 deletions = 0
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	244 for filename in list(info.keys()):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	245 if filename not in filesfound:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	246 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	247 print("Removing obsolete info entry for", end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	248 print(repr(filename), "in", repr(localdir or "."))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	249 del info[filename]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	250 deletions = deletions + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	251 if deletions:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	252 writedict(info, infofilename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	253 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	254 # Remove local files that are no longer in the remote directory
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	255 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	256 if not localdir: names = os.listdir(os.curdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	257 else: names = os.listdir(localdir)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	258 except OSError:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	259 names = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	260 for name in names:
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	261 if name[0] == '.' or name in info or name in subdirs:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	262 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	263 skip = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	264 for pat in skippats:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	265 if fnmatch(name, pat):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	266 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	267 print('Skip pattern', repr(pat), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	268 print('matches', repr(name))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	269 skip = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	270 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	271 if skip:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	272 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	273 fullname = os.path.join(localdir, name)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	274 if not rmok:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	275 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	276 print('Local file', repr(fullname), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	277 print('is no longer pertinent')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	278 continue
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	279 if verbose: print('Removing local file/dir', repr(fullname))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	280 remove(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	281 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	282 # Recursively mirror subdirectories
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	283 for subdir in subdirs:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	284 if interactive:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	285 doit = askabout('subdirectory', subdir, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	286 if not doit: continue
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	287 if verbose: print('Processing subdirectory', repr(subdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	288 localsubdir = os.path.join(localdir, subdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	289 pwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	290 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	291 print('Remote directory now:', repr(pwd))
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	292 print('Remote cwd', repr(subdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	293 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	294 f.cwd(subdir)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	295 except ftplib.error_perm as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	296 print("Can't chdir to", repr(subdir), ":", repr(msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	297 else:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	298 if verbose: print('Mirroring as', repr(localsubdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	299 mirrorsubdir(f, localsubdir)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	300 if verbose > 1: print('Remote cwd ..')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	301 f.cwd('..')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	302 newpwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	303 if newpwd != pwd:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	304 print('Ended up in wrong directory after cd + cd ..')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	305 print('Giving up now.')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	306 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	307 else:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	308 if verbose > 1: print('OK.')
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	309
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	310 # Helper to remove a file or directory tree
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	311 def remove(fullname):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	312 if os.path.isdir(fullname) and not os.path.islink(fullname):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	313 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	314 names = os.listdir(fullname)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	315 except OSError:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	316 names = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	317 ok = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	318 for name in names:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	319 if not remove(os.path.join(fullname, name)):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	320 ok = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	321 if not ok:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	322 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	323 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	324 os.rmdir(fullname)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	325 except OSError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	326 print("Can't remove local directory %r: %s" % (fullname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	327 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	328 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	329 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	330 os.unlink(fullname)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	331 except OSError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	332 print("Can't remove local file %r: %s" % (fullname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	333 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	334 return 1
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	335
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	336 # Wrapper around a file for writing to write a hash sign every block.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	337 class LoggingFile:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	338 def __init__(self, fp, blocksize, outfp):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	339 self.fp = fp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	340 self.bytes = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	341 self.hashes = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	342 self.blocksize = blocksize
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	343 self.outfp = outfp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	344 def write(self, data):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	345 self.bytes = self.bytes + len(data)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	346 hashes = int(self.bytes) / self.blocksize
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	347 while hashes > self.hashes:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	348 self.outfp.write('#')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	349 self.outfp.flush()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	350 self.hashes = self.hashes + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	351 self.fp.write(data)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	352 def close(self):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	353 self.outfp.write('\n')
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	354
37045 081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	355 def raw_input(prompt):
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	356 sys.stdout.write(prompt)
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	357 sys.stdout.flush()
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	358 return sys.stdin.readline()
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	359
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	360 # Ask permission to download a file.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	361 def askabout(filetype, filename, pwd):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	362 prompt = 'Retrieve %s %s from %s ? [ny] ' % (filetype, filename, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	363 while 1:
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	364 reply = raw_input(prompt).strip().lower()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	365 if reply in ['y', 'ye', 'yes']:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	366 return 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	367 if reply in ['', 'n', 'no', 'nop', 'nope']:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	368 return 0
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	369 print('Please answer yes or no.')
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	370
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	371 # Create a directory if it doesn't exist. Recursively create the
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	372 # parent directory as well if needed.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	373 def makedir(pathname):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	374 if os.path.isdir(pathname):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	375 return
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	376 dirname = os.path.dirname(pathname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	377 if dirname: makedir(dirname)
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 40587 diff changeset	378 os.mkdir(pathname, 0o777)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	379
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	380 # Write a dictionary to a file in a way that can be read back using
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	381 # rval() but is still somewhat readable (i.e. not a single long line).
8591 2f43b61a98df Fix by Sjoerd Mullender to support symbolic links and make a backup of Guido van Rossum <guido@python.org> parents: 8320 diff changeset	382 # Also creates a backup file.
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	383 def writedict(dict, filename):
29107 93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	384 dir, fname = os.path.split(filename)
93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	385 tempname = os.path.join(dir, '@' + fname)
93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	386 backup = os.path.join(dir, fname + '~')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	387 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	388 os.unlink(backup)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	389 except OSError:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	390 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	391 fp = open(tempname, 'w')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	392 fp.write('{\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	393 for key, value in dict.items():
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	394 fp.write('%r: %r,\n' % (key, value))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	395 fp.write('}\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	396 fp.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	397 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	398 os.rename(filename, backup)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 59522 diff changeset	399 except OSError:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	400 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	401 os.rename(tempname, filename)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	402
14648 15cbe399a9c3 Allow this script to act like a module by only calling main() if Barry Warsaw <barry@python.org> parents: 8591 diff changeset	403
15cbe399a9c3 Allow this script to act like a module by only calling main() if Barry Warsaw <barry@python.org> parents: 8591 diff changeset	404 if __name__ == '__main__':
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	405 main()

Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/ftpmirror.py @ 87690:66c1c9f32567 default tip