Mercurial > cpython-lru_cache-weakref
annotate Tools/scripts/findnocoding.py @ 73323:dad02a080bbc
Improve byte-compilation in packaging to be independent of -O or -B.
The code I fixed to comply with PEP 3147 still had one bug: When run
under python -O, some paths for pyc files would be pyo, because I called
imp.cache_from_source without explicit debug_override argument in some
places, and under -O that would return .pyo (this is well explained in
the imp docs). Now all code (util.byte_compile, build_py, install_lib)
can create .pyo files according to options given by users,
without interference from the calling Python’s own optimize mode.
On a related topic, I also removed the code that prevented byte
compilation under python -B. The rationale is that packaging gives
control over the creation of pyc files to the user with its own explicit
option, and the behavior should not be changed if the calling Python
happens to run with -B for whatever reason. I will argue that this is a
bug fix and ask to be allowed to backport this change to distutils.
Finally, I moved one nugget of information about the --compile and
--optimize options from the source into the doc. It clears up a
misunderstanding that I (and maybe other people) had.
| author | Éric Araujo <merwok@netwok.org> |
|---|---|
| date | Thu, 03 Nov 2011 05:08:28 +0100 |
| parents | 6d9cb906c85e |
| children | 696cb524322a |
| rev | line source |
|---|---|
|
59522
fcfe324663a6
convert shebang lines: python -> python3
Benjamin Peterson <benjamin@python.org>
parents:
49633
diff
changeset
|
1 #!/usr/bin/env python3 |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
2 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
3 """List all those Python files that require a coding directive |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
4 |
|
70404
6d9cb906c85e
Remove obsolete 'rU' mode in Tools/scripts too (see be5b8d1ded34)
Éric Araujo <merwok@netwok.org>
parents:
59522
diff
changeset
|
5 Usage: findnocoding.py dir1 [dir2...] |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
6 """ |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
7 |
|
40450
cc992d75d5b3
Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents:
35465
diff
changeset
|
8 __author__ = "Oleg Broytmann, Georg Brandl" |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
9 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
10 import sys, os, re, getopt |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
11 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
12 # our pysource module finds Python source files |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
13 try: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
14 import pysource |
|
49633
bd0e9a9cd283
Merged revisions 67028,67040,67044,67046,67052,67065,67070,67077,67082 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
49591
diff
changeset
|
15 except ImportError: |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
16 # emulate the module with a simple os.walk |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
17 class pysource: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
18 has_python_ext = looks_like_python = can_be_compiled = None |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
19 def walk_python_files(self, paths, *args, **kwargs): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
20 for path in paths: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
21 if os.path.isfile(path): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
22 yield path.endswith(".py") |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
23 elif os.path.isdir(path): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
24 for root, dirs, files in os.walk(path): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
25 for filename in files: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
26 if filename.endswith(".py"): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
27 yield os.path.join(root, filename) |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
28 pysource = pysource() |
|
35465
3e5c07faab11
Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents:
35424
diff
changeset
|
29 |
|
3e5c07faab11
Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents:
35424
diff
changeset
|
30 |
|
42427
46625e6122e9
Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents:
40587
diff
changeset
|
31 print("The pysource module is not available; " |
|
46625e6122e9
Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents:
40587
diff
changeset
|
32 "no sophisticated Python source file search will be done.", file=sys.stderr) |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
33 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
34 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
35 decl_re = re.compile(r"coding[=:]\s*([-\w.]+)") |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
36 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
37 def get_declaration(line): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
38 match = decl_re.search(line) |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
39 if match: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
40 return match.group(1) |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
41 return '' |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
42 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
43 def has_correct_encoding(text, codec): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
44 try: |
|
47100
893a4dc17a91
Ran 2to3 over scripts directory.
Georg Brandl <georg@python.org>
parents:
42427
diff
changeset
|
45 str(text, codec) |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
46 except UnicodeDecodeError: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
47 return False |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
48 else: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
49 return True |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
50 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
51 def needs_declaration(fullpath): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
52 try: |
|
70404
6d9cb906c85e
Remove obsolete 'rU' mode in Tools/scripts too (see be5b8d1ded34)
Éric Araujo <merwok@netwok.org>
parents:
59522
diff
changeset
|
53 infile = open(fullpath) |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
54 except IOError: # Oops, the file was removed - ignore it |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
55 return None |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
56 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
57 line1 = infile.readline() |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
58 line2 = infile.readline() |
|
35465
3e5c07faab11
Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents:
35424
diff
changeset
|
59 |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
60 if get_declaration(line1) or get_declaration(line2): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
61 # the file does have an encoding declaration, so trust it |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
62 infile.close() |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
63 return False |
|
35465
3e5c07faab11
Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents:
35424
diff
changeset
|
64 |
|
49591
860737c96e97
default source encoding is now utf-8
Benjamin Peterson <benjamin@python.org>
parents:
47100
diff
changeset
|
65 # check the whole file for non utf-8 characters |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
66 rest = infile.read() |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
67 infile.close() |
|
35465
3e5c07faab11
Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents:
35424
diff
changeset
|
68 |
|
49591
860737c96e97
default source encoding is now utf-8
Benjamin Peterson <benjamin@python.org>
parents:
47100
diff
changeset
|
69 if has_correct_encoding(line1+line2+rest, "utf-8"): |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
70 return False |
|
35465
3e5c07faab11
Whitespace normalization (via reindent.py).
Tim Peters <tim.peters@gmail.com>
parents:
35424
diff
changeset
|
71 |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
72 return True |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
73 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
74 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
75 usage = """Usage: %s [-cd] paths... |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
76 -c: recognize Python source files trying to compile them |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
77 -d: debug output""" % sys.argv[0] |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
78 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
79 try: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
80 opts, args = getopt.getopt(sys.argv[1:], 'cd') |
|
40587
6b18a095c2e7
SF patch 1631942 by Collin Winter:
Guido van Rossum <guido@python.org>
parents:
40450
diff
changeset
|
81 except getopt.error as msg: |
|
42427
46625e6122e9
Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents:
40587
diff
changeset
|
82 print(msg, file=sys.stderr) |
|
46625e6122e9
Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents:
40587
diff
changeset
|
83 print(usage, file=sys.stderr) |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
84 sys.exit(1) |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
85 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
86 is_python = pysource.looks_like_python |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
87 debug = False |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
88 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
89 for o, a in opts: |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
90 if o == '-c': |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
91 is_python = pysource.can_be_compiled |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
92 elif o == '-d': |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
93 debug = True |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
94 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
95 if not args: |
|
42427
46625e6122e9
Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents:
40587
diff
changeset
|
96 print(usage, file=sys.stderr) |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
97 sys.exit(1) |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
98 |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
99 for fullpath in pysource.walk_python_files(args, is_python): |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
100 if debug: |
|
42427
46625e6122e9
Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents:
40587
diff
changeset
|
101 print("Testing for coding: %s" % fullpath) |
|
35424
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
102 result = needs_declaration(fullpath) |
|
7caabc749975
Patch [ 784089 ] A program to scan python files and list those require coding
Georg Brandl <georg@python.org>
parents:
diff
changeset
|
103 if result: |
|
42427
46625e6122e9
Convert print statements to function calls in Tools/.
Collin Winter <collinw@gmail.com>
parents:
40587
diff
changeset
|
104 print(fullpath) |
