cpython-lru_cache-weakref: Tools/scripts/fixcid.py annotate

annotate Tools/scripts/fixcid.py @ 87690:66c1c9f32567 default tip

Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859

author	Radomir Dopieralski <devel@sheep.art.pl>
date	Tue, 03 Dec 2013 16:02:48 +0100
parents	6cfe2982de42
children

rev	line source
59522 fcfe324663a6 convert shebang lines: python -> python3 Benjamin Peterson <benjamin@python.org> parents: 47100 diff changeset	1 #! /usr/bin/env python3
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	2
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	3 # Perform massive identifier substitution on C source files.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	4 # This actually tokenizes the files (to some extent) so it can
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	5 # avoid making substitutions inside strings or comments.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	6 # Inside strings, substitutions are never made; inside comments,
5537 b1df938655e1 Correct comment; -c is off by default. Guido van Rossum <guido@python.org> parents: 4573 diff changeset	7 # it is a user option (off by default).
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	8 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	9 # The substitutions are read from one or more files whose lines,
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	10 # when not empty, after stripping comments starting with #,
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	11 # must contain exactly two words separated by whitespace: the
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	12 # old identifier and its replacement.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	13 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	14 # The option -r reverses the sense of the substitutions (this may be
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	15 # useful to undo a particular substitution).
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	16 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	17 # If the old identifier is prefixed with a '*' (with no intervening
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	18 # whitespace), then it will not be substituted inside comments.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	19 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	20 # Command line arguments are files or directories to be processed.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	21 # Directories are searched recursively for files whose name looks
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	22 # like a C file (ends in .h or .c). The special filename '-' means
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	23 # operate in filter mode: read stdin, write stdout.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	24 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	25 # Symbolic links are always ignored (except as explicit directory
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	26 # arguments).
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	27 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	28 # The original files are kept as back-up with a "~" suffix.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	29 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	30 # Changes made are reported to stdout in a diff-like format.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	31 #
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	32 # NB: by changing only the function fixline() you can turn this
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	33 # into a program for different changes to C source files; by
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	34 # changing the function wanted() you can make a different selection of
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	35 # files.
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	36
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	37 import sys
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 33074 diff changeset	38 import re
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	39 import os
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	40 from stat import *
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	41 import getopt
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	42
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	43 err = sys.stderr.write
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	44 dbg = err
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	45 rep = sys.stdout.write
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	46
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	47 def usage():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	48 progname = sys.argv[0]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	49 err('Usage: ' + progname +
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	50 ' [-c] [-r] [-s file] ... file-or-directory ...\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	51 err('\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	52 err('-c : substitute inside comments\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	53 err('-r : reverse direction for following -s options\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	54 err('-s substfile : add a file of substitutions\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	55 err('\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	56 err('Each non-empty non-comment line in a substitution file must\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	57 err('contain exactly two words: an identifier and its replacement.\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	58 err('Comments start with a # character and end at end of line.\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	59 err('If an identifier is preceded with a *, it is not substituted\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	60 err('inside a comment even when -c is specified.\n')
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	61
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	62 def main():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	63 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	64 opts, args = getopt.getopt(sys.argv[1:], 'crs:')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37879 diff changeset	65 except getopt.error as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	66 err('Options error: ' + str(msg) + '\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	67 usage()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	68 sys.exit(2)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	69 bad = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	70 if not args: # No arguments
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	71 usage()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	72 sys.exit(2)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	73 for opt, arg in opts:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	74 if opt == '-c':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	75 setdocomments()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	76 if opt == '-r':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	77 setreverse()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	78 if opt == '-s':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	79 addsubst(arg)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	80 for arg in args:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	81 if os.path.isdir(arg):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	82 if recursedown(arg): bad = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	83 elif os.path.islink(arg):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	84 err(arg + ': will not process symbolic links\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	85 bad = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	86 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	87 if fix(arg): bad = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	88 sys.exit(bad)
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	89
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	90 # Change this regular expression to select a different set of files
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	91 Wanted = '^[a-zA-Z0-9_]+\.[ch]$'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	92 def wanted(name):
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 33074 diff changeset	93 return re.match(Wanted, name) >= 0
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	94
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	95 def recursedown(dirname):
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 25219 diff changeset	96 dbg('recursedown(%r)\n' % (dirname,))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	97 bad = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	98 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	99 names = os.listdir(dirname)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 76106 diff changeset	100 except OSError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	101 err(dirname + ': cannot list directory: ' + str(msg) + '\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	102 return 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	103 names.sort()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	104 subdirs = []
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	105 for name in names:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	106 if name in (os.curdir, os.pardir): continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	107 fullname = os.path.join(dirname, name)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	108 if os.path.islink(fullname): pass
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	109 elif os.path.isdir(fullname):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	110 subdirs.append(fullname)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	111 elif wanted(name):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	112 if fix(fullname): bad = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	113 for fullname in subdirs:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	114 if recursedown(fullname): bad = 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	115 return bad
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	116
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	117 def fix(filename):
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 25219 diff changeset	118 ## dbg('fix(%r)\n' % (filename,))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	119 if filename == '-':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	120 # Filter mode
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	121 f = sys.stdin
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	122 g = sys.stdout
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	123 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	124 # File replacement mode
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	125 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	126 f = open(filename, 'r')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37879 diff changeset	127 except IOError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	128 err(filename + ': cannot open: ' + str(msg) + '\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	129 return 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	130 head, tail = os.path.split(filename)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	131 tempname = os.path.join(head, '@' + tail)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	132 g = None
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	133 # If we find a match, we rewind the file and start over but
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	134 # now copy everything to a temp file.
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	135 lineno = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	136 initfixline()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	137 while 1:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	138 line = f.readline()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	139 if not line: break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	140 lineno = lineno + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	141 while line[-2:] == '\\\n':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	142 nextline = f.readline()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	143 if not nextline: break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	144 line = line + nextline
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	145 lineno = lineno + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	146 newline = fixline(line)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	147 if newline != line:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	148 if g is None:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	149 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	150 g = open(tempname, 'w')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37879 diff changeset	151 except IOError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	152 f.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	153 err(tempname+': cannot create: '+
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	154 str(msg)+'\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	155 return 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	156 f.seek(0)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	157 lineno = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	158 initfixline()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	159 rep(filename + ':\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	160 continue # restart from the beginning
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 25219 diff changeset	161 rep(repr(lineno) + '\n')
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	162 rep('< ' + line)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	163 rep('> ' + newline)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	164 if g is not None:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	165 g.write(newline)
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	166
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	167 # End of file
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	168 if filename == '-': return 0 # Done in filter mode
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	169 f.close()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	170 if not g: return 0 # No changes
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	171
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	172 # Finishing touch -- move files
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	173
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	174 # First copy the file's mode to the temp file
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	175 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	176 statbuf = os.stat(filename)
42084 229c28cb9afb Merged revisions 55817-55961 via svnmerge from Guido van Rossum <guido@python.org> parents: 40587 diff changeset	177 os.chmod(tempname, statbuf[ST_MODE] & 0o7777)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 76106 diff changeset	178 except OSError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	179 err(tempname + ': warning: chmod failed (' + str(msg) + ')\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	180 # Then make a backup of the original file as filename~
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	181 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	182 os.rename(filename, filename + '~')
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 76106 diff changeset	183 except OSError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	184 err(filename + ': warning: backup failed (' + str(msg) + ')\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	185 # Now move the temp file to the original file
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	186 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	187 os.rename(tempname, filename)
81017 6cfe2982de42 Use OESeeror instead of os.error (#16720) Andrew Svetlov <andrew.svetlov@gmail.com> parents: 76106 diff changeset	188 except OSError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	189 err(filename + ': rename failed (' + str(msg) + ')\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	190 return 1
68585 bf94b6a73fc8 #11565: Fix several typos. Patch by Piotr Kasprzyk. Ezio Melotti <ezio.melotti@gmail.com> parents: 47100 diff changeset	191 # Return success
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	192 return 0
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	193
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	194 # Tokenizing ANSI C (partly)
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	195
1928 a8b7ee0e572d Get rid of freeze (now its own directory). Guido van Rossum <guido@python.org> parents: 1506 diff changeset	196 Identifier = '$struct $?[a-zA-Z_][a-zA-Z0-9_]+'
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	197 String = '"$[^\n\\"]\\|\\\\.$*"'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	198 Char = '\'$[^\n\\\']\\|\\\\.$*\''
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	199 CommentStart = '/\*'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	200 CommentEnd = '\*/'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	201
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	202 Hexnumber = '0[xX][0-9a-fA-F][uUlL]'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	203 Octnumber = '0[0-7][uUlL]'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	204 Decnumber = '[1-9][0-9][uUlL]'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	205 Intnumber = Hexnumber + '\\|' + Octnumber + '\\|' + Decnumber
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	206 Exponent = '[eE][-+]?[0-9]+'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	207 Pointfloat = '$[0-9]+\.[0-9]*\\|\.[0-9]+$$' + Exponent + '$?'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	208 Expfloat = '[0-9]+' + Exponent
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	209 Floatnumber = Pointfloat + '\\|' + Expfloat
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	210 Number = Floatnumber + '\\|' + Intnumber
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	211
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	212 # Anything else is an operator -- don't list this explicitly because of '/*'
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	213
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	214 OutsideComment = (Identifier, Number, String, Char, CommentStart)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 33074 diff changeset	215 OutsideCommentPattern = '(' + '\|'.join(OutsideComment) + ')'
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 33074 diff changeset	216 OutsideCommentProgram = re.compile(OutsideCommentPattern)
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	217
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	218 InsideComment = (Identifier, Number, CommentEnd)
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 33074 diff changeset	219 InsideCommentPattern = '(' + '\|'.join(InsideComment) + ')'
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 33074 diff changeset	220 InsideCommentProgram = re.compile(InsideCommentPattern)
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	221
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	222 def initfixline():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	223 global Program
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	224 Program = OutsideCommentProgram
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	225
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	226 def fixline(line):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	227 global Program
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 25219 diff changeset	228 ## print '-->', repr(line)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	229 i = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	230 while i < len(line):
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	231 i = Program.search(line, i)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	232 if i < 0: break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	233 found = Program.group(0)
15932 1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	234 ## if Program is InsideCommentProgram: print '...',
1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	235 ## else: print ' ',
1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	236 ## print found
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	237 if len(found) == 2:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	238 if found == '/*':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	239 Program = InsideCommentProgram
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	240 elif found == '*/':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	241 Program = OutsideCommentProgram
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	242 n = len(found)
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	243 if found in Dict:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	244 subst = Dict[found]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	245 if Program is InsideCommentProgram:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	246 if not Docomments:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 42084 diff changeset	247 print('Found in comment:', found)
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	248 i = i + n
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	249 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	250 if NotInComment.has_key(found):
15932 1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	251 ## print 'Ignored in comment:',
1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	252 ## print found, '-->', subst
1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	253 ## print 'Line:', line,
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	254 subst = found
15932 1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	255 ## else:
1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	256 ## print 'Substituting in comment:',
1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	257 ## print found, '-->', subst
1881b968989b Hand repair of cases where reindent changed lines of the form Tim Peters <tim.peters@gmail.com> parents: 15930 diff changeset	258 ## print 'Line:', line,
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	259 line = line[:i] + subst + line[i+n:]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	260 n = len(subst)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	261 i = i + n
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	262 return line
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	263
1502 8965ef4eb399 Various changes, andded -c option Guido van Rossum <guido@python.org> parents: 1309 diff changeset	264 Docomments = 0
8965ef4eb399 Various changes, andded -c option Guido van Rossum <guido@python.org> parents: 1309 diff changeset	265 def setdocomments():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	266 global Docomments
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	267 Docomments = 1
1502 8965ef4eb399 Various changes, andded -c option Guido van Rossum <guido@python.org> parents: 1309 diff changeset	268
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	269 Reverse = 0
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	270 def setreverse():
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	271 global Reverse
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	272 Reverse = (not Reverse)
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	273
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	274 Dict = {}
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	275 NotInComment = {}
02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	276 def addsubst(substfile):
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	277 try:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	278 fp = open(substfile, 'r')
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 37879 diff changeset	279 except IOError as msg:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	280 err(substfile + ': cannot read substfile: ' + str(msg) + '\n')
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	281 sys.exit(1)
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	282 lineno = 0
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	283 while 1:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	284 line = fp.readline()
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	285 if not line: break
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	286 lineno = lineno + 1
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	287 try:
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 15932 diff changeset	288 i = line.index('#')
8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 15932 diff changeset	289 except ValueError:
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	290 i = -1 # Happens to delete trailing \n
25219 8a7a2f1f88f0 Apply diff2.txt from SF patch http://www.python.org/sf/572113 Walter Dörwald <walter@livinglogic.de> parents: 15932 diff changeset	291 words = line[:i].split()
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	292 if not words: continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	293 if len(words) == 3 and words[0] == 'struct':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	294 words[:2] = [words[0] + ' ' + words[1]]
76106 62dde5dd475e #14490, #14491: add 'sundry'-style import tests for Tools/scripts. R David Murray <rdmurray@bitdance.com> parents: 68586 diff changeset	295 elif len(words) != 2:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 25219 diff changeset	296 err(substfile + '%s:%r: warning: bad line: %r' % (substfile, lineno, line))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	297 continue
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	298 if Reverse:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	299 [value, key] = words
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	300 else:
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	301 [key, value] = words
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	302 if value[0] == '*':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	303 value = value[1:]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	304 if key[0] == '*':
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	305 key = key[1:]
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	306 NotInComment[key] = value
47100 893a4dc17a91 Ran 2to3 over scripts directory. Georg Brandl <georg@python.org> parents: 42427 diff changeset	307 if key in Dict:
31515 71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 25219 diff changeset	308 err('%s:%r: warning: overriding: %r %r\n' % (substfile, lineno, key, value))
71a294bd7950 Replace backticks with repr() or "%r" Walter Dörwald <walter@livinglogic.de> parents: 25219 diff changeset	309 err('%s:%r: warning: previous: %r\n' % (substfile, lineno, Dict[key]))
15930 14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	310 Dict[key] = value
14a27acc755a Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 5537 diff changeset	311 fp.close()
1309 02c03afc513e Initial revision Guido van Rossum <guido@python.org> parents: diff changeset	312
33074 981fff7363b9 [Patch #1005491 ] use __name__ == '__main__' in scripts Andrew M. Kuchling <amk@amk.ca> parents: 31515 diff changeset	313 if __name__ == '__main__':
981fff7363b9 [Patch #1005491 ] use __name__ == '__main__' in scripts Andrew M. Kuchling <amk@amk.ca> parents: 31515 diff changeset	314 main()

Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/fixcid.py @ 87690:66c1c9f32567 default tip