cpython-lru_cache-weakref: Tools/scripts/ftpmirror.py annotate

annotate Tools/scripts/ftpmirror.py @ 70817:dd775b1561ba

Fix markup.

author	Georg Brandl <georg@python.org>
date	Tue, 14 Jun 2011 21:09:55 +0200
parents	fcfe324663a6
children	6cfe2982de42

rev	line source
59522 fcfe324663a6 convert shebang lines: python -> python3 Benjamin Peterson <benjamin@python.org> parents: 47100 diff changeset	1 #! /usr/bin/env python3
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	2
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	3 """Mirror a remote ftp subtree into a local directory tree.
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	4
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	5 usage: ftpmirror [-v] [-q] [-i] [-m] [-n] [-r] [-s pat]
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	6 [-l username [-p passwd [-a account]]]
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	7 hostname[:port] [remotedir [localdir]]
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	8 -v: verbose
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	9 -q: quiet
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	10 -i: interactive mode
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	11 -m: macintosh server (NCSA telnet 2.4) (implies -n -s '*.o')
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	12 -n: don't log in
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	13 -r: remove local files/directories no longer pertinent
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	14 -l username [-p passwd [-a account]]: login info (default .netrc or anonymous)
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	15 -s pat: skip files matching pattern
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	16 hostname: remote host w/ optional port separated by ':'
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	17 remotedir: remote directory (default initial)
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	18 localdir: local directory (default current)
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	19 """
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	20
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	21 import os
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	22 import sys
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	23 import time
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	24 import getopt
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	25 import ftplib
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	26 import netrc
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	27 from fnmatch import fnmatch
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	28
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	29 # Print usage message and exit
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	30 def usage(*args):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	31 sys.stdout = sys.stderr
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	32 for msg in args: print(msg)
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	33 print(__doc__)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	34 sys.exit(2)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	35
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	36 verbose = 1 # 0 for -q, 2 for -v
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	37 interactive = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	38 mac = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	39 rmok = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	40 nologin = 0
3374 0c8821bf6db5 don't die on unreadable files etc. Guido van Rossum <guido@python.org> parents: 2761 diff changeset	41 skippats = ['.', '..', '.mirrorinfo']
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	42
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	43 # Main program: parse command line and start processing
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	44 def main():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	45 global verbose, interactive, mac, rmok, nologin
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	46 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	47 opts, args = getopt.getopt(sys.argv[1:], 'a:bil:mnp:qrs:v')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	48 except getopt.error as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	49 usage(msg)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	50 login = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	51 passwd = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	52 account = ''
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	53 if not args: usage('hostname missing')
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	54 host = args[0]
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	55 port = 0
61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	56 if ':' in host:
61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	57 host, port = host.split(':', 1)
26536 6ea786528395 * when given, port should be converted to int Skip Montanaro <skip@pobox.com> parents: 26535 diff changeset	58 port = int(port)
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	59 try:
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	60 auth = netrc.netrc().authenticators(host)
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	61 if auth is not None:
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	62 login, account, passwd = auth
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	63 except (netrc.NetrcParseError, IOError):
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	64 pass
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	65 for o, a in opts:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	66 if o == '-l': login = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	67 if o == '-p': passwd = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	68 if o == '-a': account = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	69 if o == '-v': verbose = verbose + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	70 if o == '-q': verbose = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	71 if o == '-i': interactive = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	72 if o == '-m': mac = 1; nologin = 1; skippats.append('*.o')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	73 if o == '-n': nologin = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	74 if o == '-r': rmok = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	75 if o == '-s': skippats.append(a)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	76 remotedir = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	77 localdir = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	78 if args[1:]:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	79 remotedir = args[1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	80 if args[2:]:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	81 localdir = args[2]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	82 if args[3:]: usage('too many arguments')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	83 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	84 f = ftplib.FTP()
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	85 if verbose: print("Connecting to '%s%s'..." % (host,
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	86 (port and ":%d"%port or "")))
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	87 f.connect(host,port)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	88 if not nologin:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	89 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	90 print('Logging in as %r...' % (login or 'anonymous'))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	91 f.login(login, passwd, account)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	92 if verbose: print('OK.')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	93 pwd = f.pwd()
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	94 if verbose > 1: print('PWD =', repr(pwd))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	95 if remotedir:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	96 if verbose > 1: print('cwd(%s)' % repr(remotedir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	97 f.cwd(remotedir)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	98 if verbose > 1: print('OK.')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	99 pwd = f.pwd()
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	100 if verbose > 1: print('PWD =', repr(pwd))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	101 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	102 mirrorsubdir(f, localdir)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	103
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	104 # Core logic: mirror one subdirectory (recursively)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	105 def mirrorsubdir(f, localdir):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	106 pwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	107 if localdir and not os.path.isdir(localdir):
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	108 if verbose: print('Creating local directory', repr(localdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	109 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	110 makedir(localdir)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	111 except os.error as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	112 print("Failed to establish local directory", repr(localdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	113 return
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	114 infofilename = os.path.join(localdir, '.mirrorinfo')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	115 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	116 text = open(infofilename, 'r').read()
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	117 except IOError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	118 text = '{}'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	119 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	120 info = eval(text)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	121 except (SyntaxError, NameError):
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	122 print('Bad mirror info in', repr(infofilename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	123 info = {}
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	124 subdirs = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	125 listing = []
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	126 if verbose: print('Listing remote directory %r...' % (pwd,))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	127 f.retrlines('LIST', listing.append)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	128 filesfound = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	129 for line in listing:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	130 if verbose > 1: print('-->', repr(line))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	131 if mac:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	132 # Mac listing has just filenames;
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	133 # trailing / means subdirectory
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	134 filename = line.strip()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	135 mode = '-'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	136 if filename[-1:] == '/':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	137 filename = filename[:-1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	138 mode = 'd'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	139 infostuff = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	140 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	141 # Parse, assuming a UNIX listing
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	142 words = line.split(None, 8)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	143 if len(words) < 6:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	144 if verbose > 1: print('Skipping short line')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	145 continue
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	146 filename = words[-1].lstrip()
8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	147 i = filename.find(" -> ")
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	148 if i >= 0:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	149 # words[0] had better start with 'l'...
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	150 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	151 print('Found symbolic link %r' % (filename,))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	152 linkto = filename[i+4:]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	153 filename = filename[:i]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	154 infostuff = words[-5:-1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	155 mode = words[0]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	156 skip = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	157 for pat in skippats:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	158 if fnmatch(filename, pat):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	159 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	160 print('Skip pattern', repr(pat), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	161 print('matches', repr(filename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	162 skip = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	163 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	164 if skip:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	165 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	166 if mode[0] == 'd':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	167 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	168 print('Remembering subdirectory', repr(filename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	169 subdirs.append(filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	170 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	171 filesfound.append(filename)
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	172 if filename in info and info[filename] == infostuff:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	173 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	174 print('Already have this version of',repr(filename))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	175 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	176 fullname = os.path.join(localdir, filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	177 tempname = os.path.join(localdir, '@'+filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	178 if interactive:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	179 doit = askabout('file', filename, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	180 if not doit:
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	181 if filename not in info:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	182 info[filename] = 'Not retrieved'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	183 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	184 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	185 os.unlink(tempname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	186 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	187 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	188 if mode[0] == 'l':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	189 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	190 print("Creating symlink %r -> %r" % (filename, linkto))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	191 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	192 os.symlink(linkto, tempname)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	193 except IOError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	194 print("Can't create %r: %s" % (tempname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	195 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	196 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	197 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	198 fp = open(tempname, 'wb')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	199 except IOError as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	200 print("Can't create %r: %s" % (tempname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	201 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	202 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	203 print('Retrieving %r from %r as %r...' % (filename, pwd, fullname))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	204 if verbose:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	205 fp1 = LoggingFile(fp, 1024, sys.stdout)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	206 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	207 fp1 = fp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	208 t0 = time.time()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	209 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	210 f.retrbinary('RETR ' + filename,
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	211 fp1.write, 8*1024)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	212 except ftplib.error_perm as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	213 print(msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	214 t1 = time.time()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	215 bytes = fp.tell()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	216 fp.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	217 if fp1 != fp:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	218 fp1.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	219 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	220 os.unlink(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	221 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	222 pass # Ignore the error
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	223 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	224 os.rename(tempname, fullname)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	225 except os.error as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	226 print("Can't rename %r to %r: %s" % (tempname, fullname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	227 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	228 info[filename] = infostuff
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	229 writedict(info, infofilename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	230 if verbose and mode[0] != 'l':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	231 dt = t1 - t0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	232 kbytes = bytes / 1024.0
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	233 print(int(round(kbytes)), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	234 print('Kbytes in', end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	235 print(int(round(dt)), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	236 print('seconds', end=' ')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	237 if t1 > t0:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	238 print('(~%d Kbytes/sec)' % \
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	239 int(round(kbytes/dt),))
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	240 print()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	241 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	242 # Remove files from info that are no longer remote
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	243 deletions = 0
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	244 for filename in list(info.keys()):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	245 if filename not in filesfound:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	246 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	247 print("Removing obsolete info entry for", end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	248 print(repr(filename), "in", repr(localdir or "."))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	249 del info[filename]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	250 deletions = deletions + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	251 if deletions:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	252 writedict(info, infofilename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	253 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	254 # Remove local files that are no longer in the remote directory
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	255 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	256 if not localdir: names = os.listdir(os.curdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	257 else: names = os.listdir(localdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	258 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	259 names = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	260 for name in names:
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	261 if name[0] == '.' or name in info or name in subdirs:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	262 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	263 skip = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	264 for pat in skippats:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	265 if fnmatch(name, pat):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	266 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	267 print('Skip pattern', repr(pat), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	268 print('matches', repr(name))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	269 skip = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	270 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	271 if skip:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	272 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	273 fullname = os.path.join(localdir, name)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	274 if not rmok:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	275 if verbose:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	276 print('Local file', repr(fullname), end=' ')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	277 print('is no longer pertinent')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	278 continue
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	279 if verbose: print('Removing local file/dir', repr(fullname))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	280 remove(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	281 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	282 # Recursively mirror subdirectories
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	283 for subdir in subdirs:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	284 if interactive:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	285 doit = askabout('subdirectory', subdir, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	286 if not doit: continue
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	287 if verbose: print('Processing subdirectory', repr(subdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	288 localsubdir = os.path.join(localdir, subdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	289 pwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	290 if verbose > 1:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	291 print('Remote directory now:', repr(pwd))
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	292 print('Remote cwd', repr(subdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	293 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	294 f.cwd(subdir)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	295 except ftplib.error_perm as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	296 print("Can't chdir to", repr(subdir), ":", repr(msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	297 else:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	298 if verbose: print('Mirroring as', repr(localsubdir))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	299 mirrorsubdir(f, localsubdir)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	300 if verbose > 1: print('Remote cwd ..')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	301 f.cwd('..')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	302 newpwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	303 if newpwd != pwd:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	304 print('Ended up in wrong directory after cd + cd ..')
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	305 print('Giving up now.')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	306 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	307 else:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	308 if verbose > 1: print('OK.')
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	309
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	310 # Helper to remove a file or directory tree
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	311 def remove(fullname):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	312 if os.path.isdir(fullname) and not os.path.islink(fullname):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	313 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	314 names = os.listdir(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	315 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	316 names = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	317 ok = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	318 for name in names:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	319 if not remove(os.path.join(fullname, name)):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	320 ok = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	321 if not ok:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	322 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	323 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	324 os.rmdir(fullname)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	325 except os.error as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	326 print("Can't remove local directory %r: %s" % (fullname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	327 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	328 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	329 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	330 os.unlink(fullname)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37045 diff changeset	331 except os.error as msg:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	332 print("Can't remove local file %r: %s" % (fullname, msg))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	333 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	334 return 1
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	335
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	336 # Wrapper around a file for writing to write a hash sign every block.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	337 class LoggingFile:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	338 def __init__(self, fp, blocksize, outfp):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	339 self.fp = fp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	340 self.bytes = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	341 self.hashes = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	342 self.blocksize = blocksize
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	343 self.outfp = outfp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	344 def write(self, data):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	345 self.bytes = self.bytes + len(data)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	346 hashes = int(self.bytes) / self.blocksize
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	347 while hashes > self.hashes:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	348 self.outfp.write('#')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	349 self.outfp.flush()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	350 self.hashes = self.hashes + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	351 self.fp.write(data)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	352 def close(self):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	353 self.outfp.write('\n')
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	354
37045 081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	355 def raw_input(prompt):
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	356 sys.stdout.write(prompt)
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	357 sys.stdout.flush()
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	358 return sys.stdin.readline()
081b4f0e81b7 Get rid of a bunch more raw_input references Neal Norwitz <nnorwitz@gmail.com> parents: 31515 diff changeset	359
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	360 # Ask permission to download a file.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	361 def askabout(filetype, filename, pwd):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	362 prompt = 'Retrieve %s %s from %s ? [ny] ' % (filetype, filename, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	363 while 1:
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	364 reply = raw_input(prompt).strip().lower()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	365 if reply in ['y', 'ye', 'yes']:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	366 return 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	367 if reply in ['', 'n', 'no', 'nop', 'nope']:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	368 return 0
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	369 print('Please answer yes or no.')
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	370
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	371 # Create a directory if it doesn't exist. Recursively create the
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	372 # parent directory as well if needed.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	373 def makedir(pathname):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	374 if os.path.isdir(pathname):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	375 return
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	376 dirname = os.path.dirname(pathname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	377 if dirname: makedir(dirname)
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 40587 diff changeset	378 os.mkdir(pathname, 0o777)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	379
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	380 # Write a dictionary to a file in a way that can be read back using
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	381 # rval() but is still somewhat readable (i.e. not a single long line).
8591 2f43b61a98df Fix by Sjoerd Mullender to support symbolic links and make a backup of Guido van Rossum <guido@python.org> parents: 8320 diff changeset	382 # Also creates a backup file.
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	383 def writedict(dict, filename):
29107 93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	384 dir, fname = os.path.split(filename)
93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	385 tempname = os.path.join(dir, '@' + fname)
93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	386 backup = os.path.join(dir, fname + '~')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	387 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	388 os.unlink(backup)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	389 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	390 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	391 fp = open(tempname, 'w')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	392 fp.write('{\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	393 for key, value in dict.items():
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	394 fp.write('%r: %r,\n' % (key, value))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	395 fp.write('}\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	396 fp.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	397 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	398 os.rename(filename, backup)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	399 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	400 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	401 os.rename(tempname, filename)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	402
14648 15cbe399a9c3 Allow this script to act like a module by only calling main() if Barry Warsaw <barry@python.org> parents: 8591 diff changeset	403
15cbe399a9c3 Allow this script to act like a module by only calling main() if Barry Warsaw <barry@python.org> parents: 8591 diff changeset	404 if __name__ == '__main__':
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	405 main()

Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/ftpmirror.py @ 70817:dd775b1561ba