annotate Tools/scripts/parseentities.py @ 73323:dad02a080bbc

Improve byte-compilation in packaging to be independent of -O or -B. The code I fixed to comply with PEP 3147 still had one bug: When run under python -O, some paths for pyc files would be pyo, because I called imp.cache_from_source without explicit debug_override argument in some places, and under -O that would return .pyo (this is well explained in the imp docs). Now all code (util.byte_compile, build_py, install_lib) can create .pyo files according to options given by users, without interference from the calling Python’s own optimize mode. On a related topic, I also removed the code that prevented byte compilation under python -B. The rationale is that packaging gives control over the creation of pyc files to the user with its own explicit option, and the behavior should not be changed if the calling Python happens to run with -B for whatever reason. I will argue that this is a bug fix and ask to be allowed to backport this change to distutils. Finally, I moved one nugget of information about the --compile and --optimize options from the source into the doc. It clears up a misunderstanding that I (and maybe other people) had.
author Éric Araujo <merwok@netwok.org>
date Thu, 03 Nov 2011 05:08:28 +0100
parents fcfe324663a6
children 62dde5dd475e
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
59522
fcfe324663a6 convert shebang lines: python -> python3
Benjamin Peterson <benjamin@python.org>
parents: 47100
diff changeset
1 #!/usr/bin/env python3
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
2 """ Utility for parsing HTML entity definitions available from:
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
3
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
4 http://www.w3.org/ as e.g.
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 http://www.w3.org/TR/REC-html40/HTMLlat1.ent
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 Input is read from stdin, output is written to stdout in form of a
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
8 Python snippet defining a dictionary "entitydefs" mapping literal
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 entity name to character or numeric entity.
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
15930
14a27acc755a Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 14657
diff changeset
11 Marc-Andre Lemburg, mal@lemburg.com, 1999.
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 Use as you like. NO WARRANTIES.
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 """
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
15 import re,sys
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
16 import TextTools
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
17
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
18 entityRE = re.compile('<!ENTITY +(\w+) +CDATA +"([^"]+)" +-- +((?:.|\n)+?) *-->')
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
19
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
20 def parse(text,pos=0,endpos=None):
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
21
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
22 pos = 0
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
23 if endpos is None:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
24 endpos = len(text)
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
25 d = {}
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
26 while 1:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
27 m = entityRE.search(text,pos,endpos)
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
28 if not m:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
29 break
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
30 name,charcode,comment = m.groups()
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
31 d[name] = charcode,comment
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
32 pos = m.end()
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
33 return d
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
34
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
35 def writefile(f,defs):
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
36
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
37 f.write("entitydefs = {\n")
47100
893a4dc17a91 Ran 2to3 over scripts directory.
Georg Brandl <georg@python.org>
parents: 15930
diff changeset
38 items = sorted(defs.items())
893a4dc17a91 Ran 2to3 over scripts directory.
Georg Brandl <georg@python.org>
parents: 15930
diff changeset
39 for name, (charcode,comment) in items:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
40 if charcode[:2] == '&#':
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
41 code = int(charcode[2:-1])
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
42 if code < 256:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
43 charcode = "'\%o'" % code
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
44 else:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
45 charcode = repr(charcode)
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
46 else:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
47 charcode = repr(charcode)
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
48 comment = TextTools.collapse(comment)
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
49 f.write(" '%s':\t%s, \t# %s\n" % (name,charcode,comment))
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
50 f.write('\n}\n')
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
51
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 if __name__ == '__main__':
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
53 if len(sys.argv) > 1:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
54 infile = open(sys.argv[1])
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 else:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
56 infile = sys.stdin
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
57 if len(sys.argv) > 2:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
58 outfile = open(sys.argv[2],'w')
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
59 else:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
60 outfile = sys.stdout
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
61 text = infile.read()
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 defs = parse(text)
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
63 writefile(outfile,defs)