cpython-lru_cache-weakref: Tools/scripts/ftpmirror.py annotate

annotate Tools/scripts/ftpmirror.py @ 80661:94de77bd0b4b 2.7

Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.

author	Andrew Svetlov <andrew.svetlov@gmail.com>
date	Thu, 29 Nov 2012 19:51:28 +0200
parents	71a294bd7950
children	081b4f0e81b7

rev	line source
4573 73758db2f464 /usr/local/bin/python -> /usr/bin/env python Guido van Rossum <guido@python.org> parents: 4544 diff changeset	1 #! /usr/bin/env python
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	2
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	3 """Mirror a remote ftp subtree into a local directory tree.
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	4
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	5 usage: ftpmirror [-v] [-q] [-i] [-m] [-n] [-r] [-s pat]
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	6 [-l username [-p passwd [-a account]]]
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	7 hostname[:port] [remotedir [localdir]]
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	8 -v: verbose
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	9 -q: quiet
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	10 -i: interactive mode
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	11 -m: macintosh server (NCSA telnet 2.4) (implies -n -s '*.o')
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	12 -n: don't log in
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	13 -r: remove local files/directories no longer pertinent
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	14 -l username [-p passwd [-a account]]: login info (default .netrc or anonymous)
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	15 -s pat: skip files matching pattern
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	16 hostname: remote host w/ optional port separated by ':'
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	17 remotedir: remote directory (default initial)
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	18 localdir: local directory (default current)
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	19 """
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	20
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	21 import os
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	22 import sys
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	23 import time
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	24 import getopt
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	25 import ftplib
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	26 import netrc
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	27 from fnmatch import fnmatch
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	28
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	29 # Print usage message and exit
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	30 def usage(*args):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	31 sys.stdout = sys.stderr
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	32 for msg in args: print msg
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	33 print __doc__
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	34 sys.exit(2)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	35
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	36 verbose = 1 # 0 for -q, 2 for -v
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	37 interactive = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	38 mac = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	39 rmok = 0
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	40 nologin = 0
3374 0c8821bf6db5 don't die on unreadable files etc. Guido van Rossum <guido@python.org> parents: 2761 diff changeset	41 skippats = ['.', '..', '.mirrorinfo']
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	42
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	43 # Main program: parse command line and start processing
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	44 def main():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	45 global verbose, interactive, mac, rmok, nologin
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	46 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	47 opts, args = getopt.getopt(sys.argv[1:], 'a:bil:mnp:qrs:v')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	48 except getopt.error, msg:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	49 usage(msg)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	50 login = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	51 passwd = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	52 account = ''
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	53 if not args: usage('hostname missing')
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	54 host = args[0]
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	55 port = 0
61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	56 if ':' in host:
61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	57 host, port = host.split(':', 1)
26536 6ea786528395 * when given, port should be converted to int Skip Montanaro <skip@pobox.com> parents: 26535 diff changeset	58 port = int(port)
18052 c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	59 try:
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	60 auth = netrc.netrc().authenticators(host)
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	61 if auth is not None:
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	62 login, account, passwd = auth
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	63 except (netrc.NetrcParseError, IOError):
c4016abd6b7f Patch #430754: Makes ftpmirror.py .netrc aware Martin v. Löwis <martin@v.loewis.de> parents: 15930 diff changeset	64 pass
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	65 for o, a in opts:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	66 if o == '-l': login = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	67 if o == '-p': passwd = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	68 if o == '-a': account = a
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	69 if o == '-v': verbose = verbose + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	70 if o == '-q': verbose = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	71 if o == '-i': interactive = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	72 if o == '-m': mac = 1; nologin = 1; skippats.append('*.o')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	73 if o == '-n': nologin = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	74 if o == '-r': rmok = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	75 if o == '-s': skippats.append(a)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	76 remotedir = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	77 localdir = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	78 if args[1:]:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	79 remotedir = args[1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	80 if args[2:]:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	81 localdir = args[2]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	82 if args[3:]: usage('too many arguments')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	83 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	84 f = ftplib.FTP()
26536 6ea786528395 * when given, port should be converted to int Skip Montanaro <skip@pobox.com> parents: 26535 diff changeset	85 if verbose: print "Connecting to '%s%s'..." % (host,
6ea786528395 * when given, port should be converted to int Skip Montanaro <skip@pobox.com> parents: 26535 diff changeset	86 (port and ":%d"%port or ""))
26535 61e2b8cefd86 allow optional port specified as part of the hostname, e.g., Skip Montanaro <skip@pobox.com> parents: 25219 diff changeset	87 f.connect(host,port)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	88 if not nologin:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	89 if verbose:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	90 print 'Logging in as %r...' % (login or 'anonymous')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	91 f.login(login, passwd, account)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	92 if verbose: print 'OK.'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	93 pwd = f.pwd()
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	94 if verbose > 1: print 'PWD =', repr(pwd)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	95 if remotedir:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	96 if verbose > 1: print 'cwd(%s)' % repr(remotedir)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	97 f.cwd(remotedir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	98 if verbose > 1: print 'OK.'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	99 pwd = f.pwd()
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	100 if verbose > 1: print 'PWD =', repr(pwd)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	101 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	102 mirrorsubdir(f, localdir)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	103
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	104 # Core logic: mirror one subdirectory (recursively)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	105 def mirrorsubdir(f, localdir):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	106 pwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	107 if localdir and not os.path.isdir(localdir):
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	108 if verbose: print 'Creating local directory', repr(localdir)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	109 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	110 makedir(localdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	111 except os.error, msg:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	112 print "Failed to establish local directory", repr(localdir)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	113 return
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	114 infofilename = os.path.join(localdir, '.mirrorinfo')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	115 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	116 text = open(infofilename, 'r').read()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	117 except IOError, msg:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	118 text = '{}'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	119 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	120 info = eval(text)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	121 except (SyntaxError, NameError):
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	122 print 'Bad mirror info in', repr(infofilename)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	123 info = {}
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	124 subdirs = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	125 listing = []
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	126 if verbose: print 'Listing remote directory %r...' % (pwd,)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	127 f.retrlines('LIST', listing.append)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	128 filesfound = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	129 for line in listing:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	130 if verbose > 1: print '-->', repr(line)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	131 if mac:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	132 # Mac listing has just filenames;
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	133 # trailing / means subdirectory
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	134 filename = line.strip()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	135 mode = '-'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	136 if filename[-1:] == '/':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	137 filename = filename[:-1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	138 mode = 'd'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	139 infostuff = ''
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	140 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	141 # Parse, assuming a UNIX listing
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	142 words = line.split(None, 8)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	143 if len(words) < 6:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	144 if verbose > 1: print 'Skipping short line'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	145 continue
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	146 filename = words[-1].lstrip()
8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	147 i = filename.find(" -> ")
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	148 if i >= 0:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	149 # words[0] had better start with 'l'...
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	150 if verbose > 1:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	151 print 'Found symbolic link %r' % (filename,)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	152 linkto = filename[i+4:]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	153 filename = filename[:i]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	154 infostuff = words[-5:-1]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	155 mode = words[0]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	156 skip = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	157 for pat in skippats:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	158 if fnmatch(filename, pat):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	159 if verbose > 1:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	160 print 'Skip pattern', repr(pat),
71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	161 print 'matches', repr(filename)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	162 skip = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	163 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	164 if skip:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	165 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	166 if mode[0] == 'd':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	167 if verbose > 1:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	168 print 'Remembering subdirectory', repr(filename)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	169 subdirs.append(filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	170 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	171 filesfound.append(filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	172 if info.has_key(filename) and info[filename] == infostuff:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	173 if verbose > 1:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	174 print 'Already have this version of',repr(filename)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	175 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	176 fullname = os.path.join(localdir, filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	177 tempname = os.path.join(localdir, '@'+filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	178 if interactive:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	179 doit = askabout('file', filename, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	180 if not doit:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	181 if not info.has_key(filename):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	182 info[filename] = 'Not retrieved'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	183 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	184 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	185 os.unlink(tempname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	186 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	187 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	188 if mode[0] == 'l':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	189 if verbose:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	190 print "Creating symlink %r -> %r" % (filename, linkto)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	191 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	192 os.symlink(linkto, tempname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	193 except IOError, msg:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	194 print "Can't create %r: %s" % (tempname, msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	195 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	196 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	197 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	198 fp = open(tempname, 'wb')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	199 except IOError, msg:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	200 print "Can't create %r: %s" % (tempname, msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	201 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	202 if verbose:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	203 print 'Retrieving %r from %r as %r...' % (filename, pwd, fullname)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	204 if verbose:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	205 fp1 = LoggingFile(fp, 1024, sys.stdout)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	206 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	207 fp1 = fp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	208 t0 = time.time()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	209 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	210 f.retrbinary('RETR ' + filename,
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	211 fp1.write, 8*1024)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	212 except ftplib.error_perm, msg:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	213 print msg
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	214 t1 = time.time()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	215 bytes = fp.tell()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	216 fp.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	217 if fp1 != fp:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	218 fp1.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	219 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	220 os.unlink(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	221 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	222 pass # Ignore the error
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	223 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	224 os.rename(tempname, fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	225 except os.error, msg:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	226 print "Can't rename %r to %r: %s" % (tempname, fullname, msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	227 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	228 info[filename] = infostuff
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	229 writedict(info, infofilename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	230 if verbose and mode[0] != 'l':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	231 dt = t1 - t0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	232 kbytes = bytes / 1024.0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	233 print int(round(kbytes)),
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	234 print 'Kbytes in',
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	235 print int(round(dt)),
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	236 print 'seconds',
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	237 if t1 > t0:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	238 print '(~%d Kbytes/sec)' % \
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	239 int(round(kbytes/dt),)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	240 print
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	241 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	242 # Remove files from info that are no longer remote
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	243 deletions = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	244 for filename in info.keys():
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	245 if filename not in filesfound:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	246 if verbose:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	247 print "Removing obsolete info entry for",
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	248 print repr(filename), "in", repr(localdir or ".")
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	249 del info[filename]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	250 deletions = deletions + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	251 if deletions:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	252 writedict(info, infofilename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	253 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	254 # Remove local files that are no longer in the remote directory
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	255 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	256 if not localdir: names = os.listdir(os.curdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	257 else: names = os.listdir(localdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	258 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	259 names = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	260 for name in names:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	261 if name[0] == '.' or info.has_key(name) or name in subdirs:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	262 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	263 skip = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	264 for pat in skippats:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	265 if fnmatch(name, pat):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	266 if verbose > 1:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	267 print 'Skip pattern', repr(pat),
71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	268 print 'matches', repr(name)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	269 skip = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	270 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	271 if skip:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	272 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	273 fullname = os.path.join(localdir, name)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	274 if not rmok:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	275 if verbose:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	276 print 'Local file', repr(fullname),
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	277 print 'is no longer pertinent'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	278 continue
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	279 if verbose: print 'Removing local file/dir', repr(fullname)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	280 remove(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	281 #
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	282 # Recursively mirror subdirectories
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	283 for subdir in subdirs:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	284 if interactive:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	285 doit = askabout('subdirectory', subdir, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	286 if not doit: continue
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	287 if verbose: print 'Processing subdirectory', repr(subdir)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	288 localsubdir = os.path.join(localdir, subdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	289 pwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	290 if verbose > 1:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	291 print 'Remote directory now:', repr(pwd)
71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	292 print 'Remote cwd', repr(subdir)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	293 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	294 f.cwd(subdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	295 except ftplib.error_perm, msg:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	296 print "Can't chdir to", repr(subdir), ":", repr(msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	297 else:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	298 if verbose: print 'Mirroring as', repr(localsubdir)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	299 mirrorsubdir(f, localsubdir)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	300 if verbose > 1: print 'Remote cwd ..'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	301 f.cwd('..')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	302 newpwd = f.pwd()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	303 if newpwd != pwd:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	304 print 'Ended up in wrong directory after cd + cd ..'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	305 print 'Giving up now.'
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	306 break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	307 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	308 if verbose > 1: print 'OK.'
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	309
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	310 # Helper to remove a file or directory tree
e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	311 def remove(fullname):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	312 if os.path.isdir(fullname) and not os.path.islink(fullname):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	313 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	314 names = os.listdir(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	315 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	316 names = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	317 ok = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	318 for name in names:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	319 if not remove(os.path.join(fullname, name)):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	320 ok = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	321 if not ok:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	322 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	323 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	324 os.rmdir(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	325 except os.error, msg:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	326 print "Can't remove local directory %r: %s" % (fullname, msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	327 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	328 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	329 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	330 os.unlink(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	331 except os.error, msg:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	332 print "Can't remove local file %r: %s" % (fullname, msg)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	333 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	334 return 1
7024 e17fe79305c7 Dangerous feature added: when removing local files (i.e., only when -r Guido van Rossum <guido@python.org> parents: 5546 diff changeset	335
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	336 # Wrapper around a file for writing to write a hash sign every block.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	337 class LoggingFile:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	338 def __init__(self, fp, blocksize, outfp):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	339 self.fp = fp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	340 self.bytes = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	341 self.hashes = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	342 self.blocksize = blocksize
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	343 self.outfp = outfp
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	344 def write(self, data):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	345 self.bytes = self.bytes + len(data)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	346 hashes = int(self.bytes) / self.blocksize
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	347 while hashes > self.hashes:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	348 self.outfp.write('#')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	349 self.outfp.flush()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	350 self.hashes = self.hashes + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	351 self.fp.write(data)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	352 def close(self):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	353 self.outfp.write('\n')
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	354
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	355 # Ask permission to download a file.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	356 def askabout(filetype, filename, pwd):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	357 prompt = 'Retrieve %s %s from %s ? [ny] ' % (filetype, filename, pwd)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	358 while 1:
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 18052 diff changeset	359 reply = raw_input(prompt).strip().lower()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	360 if reply in ['y', 'ye', 'yes']:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	361 return 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	362 if reply in ['', 'n', 'no', 'nop', 'nope']:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	363 return 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	364 print 'Please answer yes or no.'
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	365
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	366 # Create a directory if it doesn't exist. Recursively create the
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	367 # parent directory as well if needed.
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	368 def makedir(pathname):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	369 if os.path.isdir(pathname):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	370 return
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	371 dirname = os.path.dirname(pathname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	372 if dirname: makedir(dirname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	373 os.mkdir(pathname, 0777)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	374
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	375 # Write a dictionary to a file in a way that can be read back using
b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	376 # rval() but is still somewhat readable (i.e. not a single long line).
8591 2f43b61a98df Fix by Sjoerd Mullender to support symbolic links and make a backup of Guido van Rossum <guido@python.org> parents: 8320 diff changeset	377 # Also creates a backup file.
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	378 def writedict(dict, filename):
29107 93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	379 dir, fname = os.path.split(filename)
93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	380 tempname = os.path.join(dir, '@' + fname)
93c92f20b6bf Fix use of 'file' as a variable name. Andrew M. Kuchling <amk@amk.ca> parents: 26536 diff changeset	381 backup = os.path.join(dir, fname + '~')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	382 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	383 os.unlink(backup)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	384 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	385 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	386 fp = open(tempname, 'w')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	387 fp.write('{\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	388 for key, value in dict.items():
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 29107 diff changeset	389 fp.write('%r: %r,\n' % (key, value))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	390 fp.write('}\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	391 fp.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	392 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	393 os.rename(filename, backup)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	394 except os.error:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	395 pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	396 os.rename(tempname, filename)
2761 b40cabfdc91d added ftp mirror script Guido van Rossum <guido@python.org> parents: diff changeset	397
14648 15cbe399a9c3 Allow this script to act like a module by only calling main() if Barry Warsaw <barry@python.org> parents: 8591 diff changeset	398
15cbe399a9c3 Allow this script to act like a module by only calling main() if Barry Warsaw <barry@python.org> parents: 8591 diff changeset	399 if __name__ == '__main__':
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 14648 diff changeset	400 main()

Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/ftpmirror.py @ 80661:94de77bd0b4b 2.7