Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/findnocoding.py @ 80661:94de77bd0b4b 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16194: document imp.load_dynamic problems when read several extension modules from the same binary object file.
author Andrew Svetlov <andrew.svetlov@gmail.com>
date Thu, 29 Nov 2012 19:51:28 +0200
parents 82ccc7e77ec5
children 3d46ef0c62c5
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
1 #!/usr/bin/env python
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
2
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
3 """List all those Python files that require a coding directive
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
4
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
5 Usage: nocoding.py dir1 [dir2...]
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
6 """
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
7
40032
5aea8b1fd262 Fix name.
Georg Brandl <georg@python.org>
parents: 35465
diff changeset
8 __author__ = "Oleg Broytmann, Georg Brandl"
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
9
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
10 import sys, os, re, getopt
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
11
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
12 # our pysource module finds Python source files
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
13 try:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
14 import pysource
49677
82ccc7e77ec5 Merged revisions 66801,66803-66804,66813,66854-66856,66866,66870-66872,66874,66887,66903,66905,66911,66913,66927,66932,66938,66942,66962,66964,66973-66974,66977,66992,66998-66999,67002,67005,67007,67028,67040-67041,67044,67070,67089,67091,67101,67117-67119,67123-67124 via svnmerge from
Georg Brandl <georg@python.org>
parents: 40032
diff changeset
15 except ImportError:
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
16 # emulate the module with a simple os.walk
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
17 class pysource:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
18 has_python_ext = looks_like_python = can_be_compiled = None
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
19 def walk_python_files(self, paths, *args, **kwargs):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
20 for path in paths:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
21 if os.path.isfile(path):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
22 yield path.endswith(".py")
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
23 elif os.path.isdir(path):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
24 for root, dirs, files in os.walk(path):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
25 for filename in files:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
26 if filename.endswith(".py"):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
27 yield os.path.join(root, filename)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
28 pysource = pysource()
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
29
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
30
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
31 print >>sys.stderr, ("The pysource module is not available; "
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
32 "no sophisticated Python source file search will be done.")
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
33
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
34
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
35 decl_re = re.compile(r"coding[=:]\s*([-\w.]+)")
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
36
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
37 def get_declaration(line):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
38 match = decl_re.search(line)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
39 if match:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
40 return match.group(1)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
41 return ''
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
42
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
43 def has_correct_encoding(text, codec):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
44 try:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
45 unicode(text, codec)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
46 except UnicodeDecodeError:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
47 return False
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
48 else:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
49 return True
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
50
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
51 def needs_declaration(fullpath):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
52 try:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
53 infile = open(fullpath, 'rU')
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
54 except IOError: # Oops, the file was removed - ignore it
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
55 return None
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
56
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
57 line1 = infile.readline()
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
58 line2 = infile.readline()
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
59
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
60 if get_declaration(line1) or get_declaration(line2):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
61 # the file does have an encoding declaration, so trust it
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
62 infile.close()
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
63 return False
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
64
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
65 # check the whole file for non-ASCII characters
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
66 rest = infile.read()
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
67 infile.close()
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
68
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
69 if has_correct_encoding(line1+line2+rest, "ascii"):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
70 return False
35465
3e5c07faab11 Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents: 35424
diff changeset
71
35424
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
72 return True
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
73
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
74
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
75 usage = """Usage: %s [-cd] paths...
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
76 -c: recognize Python source files trying to compile them
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
77 -d: debug output""" % sys.argv[0]
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
78
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
79 try:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
80 opts, args = getopt.getopt(sys.argv[1:], 'cd')
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
81 except getopt.error, msg:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
82 print >>sys.stderr, msg
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
83 print >>sys.stderr, usage
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
84 sys.exit(1)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
85
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
86 is_python = pysource.looks_like_python
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
87 debug = False
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
88
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
89 for o, a in opts:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
90 if o == '-c':
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
91 is_python = pysource.can_be_compiled
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
92 elif o == '-d':
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
93 debug = True
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
94
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
95 if not args:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
96 print >>sys.stderr, usage
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
97 sys.exit(1)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
98
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
99 for fullpath in pysource.walk_python_files(args, is_python):
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
100 if debug:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
101 print "Testing for coding: %s" % fullpath
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
102 result = needs_declaration(fullpath)
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
103 if result:
7caabc749975 Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff changeset
104 print fullpath