Mercurial > cpython-lru_cache-weakref

annotate Tools/scripts/parseentities.py @ 87690:66c1c9f32567 default tip

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Add use_weakref option to functools.lru_cache The use_weakref option makes lru_cache create weak references to the function arguments when creating the cache key, which prevents them from staying in the memory forever even after all other references to them are gone. This is a proof of concept for http://bugs.python.org/issue19859
author Radomir Dopieralski <devel@sheep.art.pl>
date Tue, 03 Dec 2013 16:02:48 +0100
parents 62dde5dd475e
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
59522
fcfe324663a6 convert shebang lines: python -> python3
Benjamin Peterson <benjamin@python.org>
parents: 47100
diff changeset
1 #!/usr/bin/env python3
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
2 """ Utility for parsing HTML entity definitions available from:
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
3
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
4 http://www.w3.org/ as e.g.
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 http://www.w3.org/TR/REC-html40/HTMLlat1.ent
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
7 Input is read from stdin, output is written to stdout in form of a
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
8 Python snippet defining a dictionary "entitydefs" mapping literal
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 entity name to character or numeric entity.
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
15930
14a27acc755a Whitespace normalization.
Tim Peters <tim.peters@gmail.com>
parents: 14657
diff changeset
11 Marc-Andre Lemburg, mal@lemburg.com, 1999.
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 Use as you like. NO WARRANTIES.
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
14 """
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
15 import re,sys
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
16
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
17 entityRE = re.compile('<!ENTITY +(\w+) +CDATA +"([^"]+)" +-- +((?:.|\n)+?) *-->')
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
18
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
19 def parse(text,pos=0,endpos=None):
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
20
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
21 pos = 0
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
22 if endpos is None:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
23 endpos = len(text)
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
24 d = {}
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
25 while 1:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
26 m = entityRE.search(text,pos,endpos)
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
27 if not m:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
28 break
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
29 name,charcode,comment = m.groups()
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
30 d[name] = charcode,comment
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
31 pos = m.end()
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
32 return d
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
33
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
34 def writefile(f,defs):
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
35
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
36 f.write("entitydefs = {\n")
47100
893a4dc17a91 Ran 2to3 over scripts directory.
Georg Brandl <georg@python.org>
parents: 15930
diff changeset
37 items = sorted(defs.items())
893a4dc17a91 Ran 2to3 over scripts directory.
Georg Brandl <georg@python.org>
parents: 15930
diff changeset
38 for name, (charcode,comment) in items:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
39 if charcode[:2] == '&#':
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
40 code = int(charcode[2:-1])
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
41 if code < 256:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
42 charcode = "'\%o'" % code
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
43 else:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
44 charcode = repr(charcode)
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
45 else:
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
46 charcode = repr(charcode)
76106
62dde5dd475e #14490, #14491: add 'sundry'-style import tests for Tools/scripts.
R David Murray <rdmurray@bitdance.com>
parents: 59522
diff changeset
47 comment = ' '.join(comment.split())
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
48 f.write(" '%s':\t%s, \t# %s\n" % (name,charcode,comment))
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
49 f.write('\n}\n')
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
50
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
51 if __name__ == '__main__':
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 if len(sys.argv) > 1:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
53 infile = open(sys.argv[1])
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 else:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
55 infile = sys.stdin
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
56 if len(sys.argv) > 2:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
57 outfile = open(sys.argv[2],'w')
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 else:
14657
946709e82be0 satisfy the tabnanny
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 11210
diff changeset
59 outfile = sys.stdout
11210
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
60 text = infile.read()
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
61 defs = parse(text)
cf7cd7b04fa6 Scrit by Marc-Andre Lemburg to generate htmlentitydefs.py.
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 writefile(outfile,defs)