Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/findnocoding.py @ 74006:88198b93ff2f

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #13560: Add PyUnicode_EncodeLocale() * Use PyUnicode_EncodeLocale() in time.strftime() if wcsftime() is not available * Document my last changes in Misc/NEWS
author Victor Stinner <victor.stinner@haypocalc.com>
date Sat, 17 Dec 2011 04:13:41 +0100
parents 6d9cb906c85e
children 696cb524322a
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
59522
fcfe324663a6 convert shebang lines: python -> python3
Benjamin Peterson <benjamin@python.org>
parents: 49633
diff changeset
1 #!/usr/bin/env python3
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
2
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
3 """List all those Python files that require a coding directive
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
4
70404
6d9cb906c85e Remove obsolete 'rU' mode in Tools/scripts too (see be5b8d1ded34)
Éric Araujo <merwok@netwok.org>
parents: 59522
diff changeset
5 Usage: findnocoding.py dir1 [dir2...]
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
6 """
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
7
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 35465
diff changeset
8 __author__ = "Oleg Broytmann, Georg Brandl"
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
9
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
10 import sys, os, re, getopt
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
11
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
12 # our pysource module finds Python source files
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
13 try:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
14 import pysource
49633
bd0e9a9cd283 Merged revisions 67028,67040,67044,67046,67052,67065,67070,67077,67082 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents: 49591
diff changeset
15 except ImportError:
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
16 # emulate the module with a simple os.walk
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
17 class pysource:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
18 has_python_ext = looks_like_python = can_be_compiled = None
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
19 def walk_python_files(self, paths, *args, **kwargs):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
20 for path in paths:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
21 if os.path.isfile(path):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
22 yield path.endswith(".py")
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
23 elif os.path.isdir(path):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
24 for root, dirs, files in os.walk(path):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
25 for filename in files:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
26 if filename.endswith(".py"):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
27 yield os.path.join(root, filename)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
28 pysource = pysource()
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
29
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
30
42427
46625e6122e9 Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents: 40587
diff changeset
31 print("The pysource module is not available; "
46625e6122e9 Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents: 40587
diff changeset
32 "no sophisticated Python source file search will be done.", file=sys.stderr)
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
33
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
34
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
35 decl_re = re.compile(r"coding[=:]\s*([-\w.]+)")
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
36
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
37 def get_declaration(line):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
38 match = decl_re.search(line)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
39 if match:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
40 return match.group(1)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
41 return ''
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
42
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
43 def has_correct_encoding(text, codec):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
44 try:
47100
893a4dc17a91 Ran 2to3 over scripts directory.
Georg Brandl <georg@python.org>
parents: 42427
diff changeset
45 str(text, codec)
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
46 except UnicodeDecodeError:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
47 return False
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
48 else:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
49 return True
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
50
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
51 def needs_declaration(fullpath):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
52 try:
70404
6d9cb906c85e Remove obsolete 'rU' mode in Tools/scripts too (see be5b8d1ded34)
Éric Araujo <merwok@netwok.org>
parents: 59522
diff changeset
53 infile = open(fullpath)
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
54 except IOError: # Oops, the file was removed - ignore it
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
55 return None
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
56
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
57 line1 = infile.readline()
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
58 line2 = infile.readline()
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
59
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
60 if get_declaration(line1) or get_declaration(line2):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
61 # the file does have an encoding declaration, so trust it
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
62 infile.close()
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
63 return False
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
64
49591
860737c96e97 default source encoding is now utf-8
Benjamin Peterson <benjamin@python.org>
parents: 47100
diff changeset
65 # check the whole file for non utf-8 characters
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
66 rest = infile.read()
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
67 infile.close()
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
68
49591
860737c96e97 default source encoding is now utf-8
Benjamin Peterson <benjamin@python.org>
parents: 47100
diff changeset
69 if has_correct_encoding(line1+line2+rest, "utf-8"):
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
70 return False
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
71
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
72 return True
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
73
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
74
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
75 usage = """Usage: %s [-cd] paths...
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
76 -c: recognize Python source files trying to compile them
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
77 -d: debug output""" % sys.argv[0]
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
78
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
79 try:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
80 opts, args = getopt.getopt(sys.argv[1:], 'cd')
40587
6b18a095c2e7 SF patch 1631942 by Collin Winter:
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
81 except getopt.error as msg:
42427
46625e6122e9 Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents: 40587
diff changeset
82 print(msg, file=sys.stderr)
46625e6122e9 Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents: 40587
diff changeset
83 print(usage, file=sys.stderr)
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
84 sys.exit(1)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
85
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
86 is_python = pysource.looks_like_python
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
87 debug = False
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
88
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
89 for o, a in opts:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
90 if o == '-c':
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
91 is_python = pysource.can_be_compiled
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
92 elif o == '-d':
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
93 debug = True
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
94
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
95 if not args:
42427
46625e6122e9 Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents: 40587
diff changeset
96 print(usage, file=sys.stderr)
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
97 sys.exit(1)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
98
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
99 for fullpath in pysource.walk_python_files(args, is_python):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
100 if debug:
42427
46625e6122e9 Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents: 40587
diff changeset
101 print("Testing for coding: %s" % fullpath)
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
102 result = needs_declaration(fullpath)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
103 if result:
42427
46625e6122e9 Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents: 40587
diff changeset
104 print(fullpath)