Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/reindent.py @ 79675:c0a2294de787 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16199: Backport 2to3 typo fix from 3.2.
author Chris Jerdonek <chris.jerdonek@gmail.com>
date Thu, 11 Oct 2012 16:15:46 -0700
parents f9cf55bbe9b9
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
1 #! /usr/bin/env python
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
2
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
3 # Released to the public domain, by Tim Peters, 03 October 2000.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
4
22573
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
5 """reindent [-d][-r][-v] [ path ... ]
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
6
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
7 -d (--dryrun) Dry run. Analyze, but don't make any changes to, files.
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
8 -r (--recurse) Recurse. Search for all .py files in subdirectories too.
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
9 -n (--nobackup) No backup. Does not make a ".bak" file before reindenting.
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
10 -v (--verbose) Verbose. Print informative msgs; else no output.
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
11 -h (--help) Help. Print this usage information and exit.
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
12
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
13 Change Python (.py) files to use 4-space indents and no hard tab characters.
20258
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
14 Also trim excess spaces and tabs from ends of lines, and remove empty lines
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
15 at the end of files. Also ensure the last line ends with a newline.
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
16
22573
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
17 If no paths are given on the command line, reindent operates as a filter,
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
18 reading a single source file from standard input and writing the transformed
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
19 source to standard output. In this case, the -d, -r and -v flags are
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
20 ignored.
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
21
22573
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
22 You can pass one or more file and/or directory paths. When a directory
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
23 path, all .py files within the directory will be examined, and, if the -r
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
24 option is given, likewise recursively for subdirectories.
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
25
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
26 If output is not to standard output, reindent overwrites files in place,
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
27 renaming the originals with a .bak extension. If it finds nothing to
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
28 change, the file is left alone. If reindent does change a file, the changed
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
29 file is a fixed-point for future runs (i.e., running reindent on the
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
30 resulting .py file won't change it again).
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
31
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
32 The hard part of reindenting is figuring out what to do with comment
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
33 lines. So long as the input files get a clean bill of health from
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
34 tabnanny.py, reindent should do a good job.
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
35
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
36 The backup file is a copy of the one that is being reindented. The ".bak"
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
37 file is generated with shutil.copy(), but some corner cases regarding
71536
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
38 user/group and permissions could leave the backup file more readable than
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
39 you'd prefer. You can always use the --nobackup option to prevent this.
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
40 """
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
41
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
42 __version__ = "1"
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
43
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
44 import tokenize
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
45 import os, shutil
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
46 import sys
71536
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
47 import io
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
48
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
49 verbose = 0
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
50 recurse = 0
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
51 dryrun = 0
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
52 makebackup = True
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
53
31832
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
54 def usage(msg=None):
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
55 if msg is not None:
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
56 print >> sys.stderr, msg
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
57 print >> sys.stderr, __doc__
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
58
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
59 def errprint(*args):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
60 sep = ""
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
61 for arg in args:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
62 sys.stderr.write(sep + str(arg))
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
63 sep = " "
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
64 sys.stderr.write("\n")
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
65
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
66 def main():
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
67 import getopt
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
68 global verbose, recurse, dryrun, makebackup
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
69 try:
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
70 opts, args = getopt.getopt(sys.argv[1:], "drnvh",
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
71 ["dryrun", "recurse", "nobackup", "verbose", "help"])
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
72 except getopt.error, msg:
31832
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
73 usage(msg)
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
74 return
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
75 for o, a in opts:
31832
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
76 if o in ('-d', '--dryrun'):
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
77 dryrun += 1
31832
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
78 elif o in ('-r', '--recurse'):
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
79 recurse += 1
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
80 elif o in ('-n', '--nobackup'):
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
81 makebackup = False
31832
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
82 elif o in ('-v', '--verbose'):
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
83 verbose += 1
31832
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
84 elif o in ('-h', '--help'):
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
85 usage()
64e2f7ec5576 add usage() function, -h(elp) flag and long versions of short flags
Skip Montanaro <skip@pobox.com>
parents: 22573
diff changeset
86 return
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
87 if not args:
22573
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
88 r = Reindenter(sys.stdin)
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
89 r.run()
298802abd38d operate as a filter when no files or directories appear on the command line
Skip Montanaro <skip@pobox.com>
parents: 21996
diff changeset
90 r.write(sys.stdout)
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
91 return
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
92 for arg in args:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
93 check(arg)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
94
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
95 def check(file):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
96 if os.path.isdir(file) and not os.path.islink(file):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
97 if verbose:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
98 print "listing directory", file
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
99 names = os.listdir(file)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
100 for name in names:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
101 fullname = os.path.join(file, name)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
102 if ((recurse and os.path.isdir(fullname) and
49677
82ccc7e77ec5 Merged revisions 66801,66803-66804,66813,66854-66856,66866,66870-66872,66874,66887,66903,66905,66911,66913,66927,66932,66938,66942,66962,66964,66973-66974,66977,66992,66998-66999,67002,67005,67007,67028,67040-67041,67044,67070,67089,67091,67101,67117-67119,67123-67124 via svnmerge from
Georg Brandl <georg@python.org>
parents: 45859
diff changeset
103 not os.path.islink(fullname) and
82ccc7e77ec5 Merged revisions 66801,66803-66804,66813,66854-66856,66866,66870-66872,66874,66887,66903,66905,66911,66913,66927,66932,66938,66942,66962,66964,66973-66974,66977,66992,66998-66999,67002,67005,67007,67028,67040-67041,67044,67070,67089,67091,67101,67117-67119,67123-67124 via svnmerge from
Georg Brandl <georg@python.org>
parents: 45859
diff changeset
104 not os.path.split(fullname)[1].startswith("."))
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
105 or name.lower().endswith(".py")):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
106 check(fullname)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
107 return
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
108
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
109 if verbose:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
110 print "checking", file, "...",
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
111 try:
71536
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
112 f = io.open(file)
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
113 except IOError, msg:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
114 errprint("%s: I/O Error: %s" % (file, str(msg)))
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
115 return
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
116
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
117 r = Reindenter(f)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
118 f.close()
71536
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
119
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
120 newline = r.newlines
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
121 if isinstance(newline, tuple):
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
122 errprint("%s: mixed newlines detected; cannot process file" % file)
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
123 return
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
124
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
125 if r.run():
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
126 if verbose:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
127 print "changed."
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
128 if dryrun:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
129 print "But this is a dry run, so leaving it alone."
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
130 if not dryrun:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
131 bak = file + ".bak"
45379
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
132 if makebackup:
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
133 shutil.copyfile(file, bak)
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
134 if verbose:
08cb5abfb5b6 Now we handle different the backup copy, because of security
Facundo Batista <facundobatista@gmail.com>
parents: 31832
diff changeset
135 print "backed up", file, "to", bak
71536
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
136 f = io.open(file, "w", newline=newline)
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
137 r.write(f)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
138 f.close()
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
139 if verbose:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
140 print "wrote new", file
45859
c424099b4ce4 Add Tools/scripts/patchcheck.py. Invoked from ``make check``, the script does
Brett Cannon <bcannon@gmail.com>
parents: 45379
diff changeset
141 return True
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
142 else:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
143 if verbose:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
144 print "unchanged."
45859
c424099b4ce4 Add Tools/scripts/patchcheck.py. Invoked from ``make check``, the script does
Brett Cannon <bcannon@gmail.com>
parents: 45379
diff changeset
145 return False
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
146
20258
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
147 def _rstrip(line, JUNK='\n \t'):
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
148 """Return line stripped of trailing spaces, tabs, newlines.
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
149
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
150 Note that line.rstrip() instead also strips sundry control characters,
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
151 but at least one known Emacs user expects to keep junk like that, not
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
152 mentioning Barry by name or anything <wink>.
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
153 """
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
154
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
155 i = len(line)
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
156 while i > 0 and line[i-1] in JUNK:
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
157 i -= 1
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
158 return line[:i]
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
159
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
160 class Reindenter:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
161
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
162 def __init__(self, f):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
163 self.find_stmt = 1 # next token begins a fresh stmt?
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
164 self.level = 0 # current indent level
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
165
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
166 # Raw file lines.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
167 self.raw = f.readlines()
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
168
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
169 # File lines, rstripped & tab-expanded. Dummy at start is so
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
170 # that we can use tokenize's 1-based line numbering easily.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
171 # Note that a line is all-blank iff it's "\n".
20258
318589404b95 Changed the reindenter to strip only trailing spaces and tabs from lines,
Tim Peters <tim.peters@gmail.com>
parents: 15018
diff changeset
172 self.lines = [_rstrip(line).expandtabs() + "\n"
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
173 for line in self.raw]
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
174 self.lines.insert(0, None)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
175 self.index = 1 # index into self.lines of next line
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
176
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
177 # List of (lineno, indentlevel) pairs, one for each stmt and
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
178 # comment line. indentlevel is -1 for comment lines, as a
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
179 # signal that tokenize doesn't know what to do about them;
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
180 # indeed, they're our headache!
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
181 self.stats = []
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
182
71536
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
183 # Save the newlines found in the file so they can be used to
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
184 # create output without mutating the newlines.
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
185 self.newlines = f.newlines
f9cf55bbe9b9 Fixes #10639: reindent.py should not convert newlines
Jason R. Coombs <jaraco@jaraco.com>
parents: 49677
diff changeset
186
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
187 def run(self):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
188 tokenize.tokenize(self.getline, self.tokeneater)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
189 # Remove trailing empty lines.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
190 lines = self.lines
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
191 while lines and lines[-1] == "\n":
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
192 lines.pop()
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
193 # Sentinel.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
194 stats = self.stats
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
195 stats.append((len(lines), 0))
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
196 # Map count of leading spaces to # we want.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
197 have2want = {}
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
198 # Program after transformation.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
199 after = self.after = []
21996
0b15cf11431e backport tim_one's checkin of
Michael W. Hudson <mwh@python.net>
parents: 20258
diff changeset
200 # Copy over initial empty lines -- there's nothing to do until
0b15cf11431e backport tim_one's checkin of
Michael W. Hudson <mwh@python.net>
parents: 20258
diff changeset
201 # we see a line with *something* on it.
0b15cf11431e backport tim_one's checkin of
Michael W. Hudson <mwh@python.net>
parents: 20258
diff changeset
202 i = stats[0][0]
0b15cf11431e backport tim_one's checkin of
Michael W. Hudson <mwh@python.net>
parents: 20258
diff changeset
203 after.extend(lines[1:i])
15018
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
204 for i in range(len(stats)-1):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
205 thisstmt, thislevel = stats[i]
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
206 nextstmt = stats[i+1][0]
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
207 have = getlspace(lines[thisstmt])
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
208 want = thislevel * 4
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
209 if want < 0:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
210 # A comment line.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
211 if have:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
212 # An indented comment line. If we saw the same
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
213 # indentation before, reuse what it most recently
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
214 # mapped to.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
215 want = have2want.get(have, -1)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
216 if want < 0:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
217 # Then it probably belongs to the next real stmt.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
218 for j in xrange(i+1, len(stats)-1):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
219 jline, jlevel = stats[j]
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
220 if jlevel >= 0:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
221 if have == getlspace(lines[jline]):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
222 want = jlevel * 4
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
223 break
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
224 if want < 0: # Maybe it's a hanging
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
225 # comment like this one,
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
226 # in which case we should shift it like its base
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
227 # line got shifted.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
228 for j in xrange(i-1, -1, -1):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
229 jline, jlevel = stats[j]
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
230 if jlevel >= 0:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
231 want = have + getlspace(after[jline-1]) - \
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
232 getlspace(lines[jline])
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
233 break
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
234 if want < 0:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
235 # Still no luck -- leave it alone.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
236 want = have
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
237 else:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
238 want = 0
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
239 assert want >= 0
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
240 have2want[have] = want
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
241 diff = want - have
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
242 if diff == 0 or have == 0:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
243 after.extend(lines[thisstmt:nextstmt])
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
244 else:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
245 for line in lines[thisstmt:nextstmt]:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
246 if diff > 0:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
247 if line == "\n":
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
248 after.append(line)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
249 else:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
250 after.append(" " * diff + line)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
251 else:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
252 remove = min(getlspace(line), -diff)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
253 after.append(line[remove:])
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
254 return self.raw != self.after
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
255
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
256 def write(self, f):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
257 f.writelines(self.after)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
258
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
259 # Line-getter for tokenize.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
260 def getline(self):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
261 if self.index >= len(self.lines):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
262 line = ""
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
263 else:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
264 line = self.lines[self.index]
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
265 self.index += 1
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
266 return line
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
267
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
268 # Line-eater for tokenize.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
269 def tokeneater(self, type, token, (sline, scol), end, line,
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
270 INDENT=tokenize.INDENT,
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
271 DEDENT=tokenize.DEDENT,
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
272 NEWLINE=tokenize.NEWLINE,
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
273 COMMENT=tokenize.COMMENT,
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
274 NL=tokenize.NL):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
275
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
276 if type == NEWLINE:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
277 # A program statement, or ENDMARKER, will eventually follow,
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
278 # after some (possibly empty) run of tokens of the form
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
279 # (NL | COMMENT)* (INDENT | DEDENT+)?
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
280 self.find_stmt = 1
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
281
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
282 elif type == INDENT:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
283 self.find_stmt = 1
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
284 self.level += 1
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
285
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
286 elif type == DEDENT:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
287 self.find_stmt = 1
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
288 self.level -= 1
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
289
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
290 elif type == COMMENT:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
291 if self.find_stmt:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
292 self.stats.append((sline, -1))
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
293 # but we're still looking for a new stmt, so leave
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
294 # find_stmt alone
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
295
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
296 elif type == NL:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
297 pass
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
298
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
299 elif self.find_stmt:
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
300 # This is the first "real token" following a NEWLINE, so it
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
301 # must be the first token of the next program statement, or an
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
302 # ENDMARKER.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
303 self.find_stmt = 0
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
304 if line: # not endmarker
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
305 self.stats.append((sline, self.level))
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
306
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
307 # Count number of leading blanks.
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
308 def getlspace(line):
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
309 i, n = 0, len(line)
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
310 while i < n and line[i] == " ":
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
311 i += 1
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
312 return i
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
313
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
314 if __name__ == '__main__':
98b1fa6c1600 New tool for normalizing indentation of .py files.
Tim Peters <tim.peters@gmail.com>
parents:
diff changeset
315 main()