cpython-lru_cache-weakref: Tools/unicode/mkstringprep.py annotate

annotate Tools/unicode/mkstringprep.py @ 66978:aed2bbafa550 3.1

Blocked revisions 87324 via svnmerge ........ r87324 | r.david.murray | 2010-12-16 14:08:51 -0500 (Thu, 16 Dec 2010) | 9 lines #10719: restore messages generated on invalid compileall args Before the introduction of filename arguments to compileall it gave semi useful messages about not being able to 'list' names that weren't valid directories. This fix restores that behavior. In addition to the test for this case, the patch also adds a test for the default behavior of compileall when no arguments are provided, and fixes a bug in one of the previously added tests. ........

author	R. David Murray <rdmurray@bitdance.com>
date	Thu, 16 Dec 2010 19:10:16 +0000
parents	63741c7521aa
children	8f95d77443da

rev	line source
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	1 import re, unicodedata, sys
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	2
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	3 if sys.maxunicode == 65535:
42715 e4ade16a694f Convert raise statements in Tools/. Collin Winter <collinw@gmail.com> parents: 42427 diff changeset	4 raise RuntimeError("need UCS-4 Python")
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	5
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	6 def gen_category(cats):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	7 for i in range(0, 0x110000):
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	8 if unicodedata.category(chr(i)) in cats:
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	9 yield(i)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	10
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	11 def gen_bidirectional(cats):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	12 for i in range(0, 0x110000):
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	13 if unicodedata.bidirectional(chr(i)) in cats:
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	14 yield(i)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	15
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	16 def compact_set(l):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	17 single = []
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	18 tuple = []
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	19 prev = None
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	20 span = 0
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	21 for e in l:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	22 if prev is None:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	23 prev = e
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	24 span = 0
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	25 continue
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	26 if prev+span+1 != e:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	27 if span > 2:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	28 tuple.append((prev,prev+span+1))
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	29 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	30 for i in range(prev, prev+span+1):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	31 single.append(i)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	32 prev = e
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	33 span = 0
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	34 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	35 span += 1
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	36 if span:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	37 tuple.append((prev,prev+span+1))
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	38 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	39 single.append(prev)
41642 a6eb6acfe04a Merged revisions 55007-55179 via svnmerge from Guido van Rossum <guido@python.org> parents: 32736 diff changeset	40 tuple = " + ".join(["list(range(%d,%d))" % t for t in tuple])
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	41 if not single:
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	42 return "set(%s)" % tuple
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	43 if not tuple:
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	44 return "set(%s)" % repr(single)
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	45 return "set(%s + %s)" % (repr(single),tuple)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	46
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	47 ############## Read the tables in the RFC #######################
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	48
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	49 data = open("rfc3454.txt").readlines()
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	50
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	51 tables = []
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	52 curname = None
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	53 for l in data:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	54 l = l.strip()
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	55 if not l:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	56 continue
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	57 # Skip RFC page breaks
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	58 if l.startswith("Hoffman & Blanchet") or\
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	59 l.startswith("RFC 3454"):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	60 continue
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	61 # Find start/end lines
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	62 m = re.match("----- (Start\|End) Table ([A-Z](.[0-9])+) -----", l)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	63 if m:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	64 if m.group(1) == "Start":
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	65 if curname:
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	66 raise RuntimeError("Double Start", (curname, l))
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	67 curname = m.group(2)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	68 table = {}
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	69 tables.append((curname, table))
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	70 continue
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	71 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	72 if not curname:
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	73 raise RuntimeError("End without start", l)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	74 curname = None
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	75 continue
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	76 if not curname:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	77 continue
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	78 # Now we are in a table
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	79 fields = l.split(";")
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	80 if len(fields) > 1:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	81 # Drop comment field
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	82 fields = fields[:-1]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	83 if len(fields) == 1:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	84 fields = fields[0].split("-")
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	85 if len(fields) > 1:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	86 # range
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	87 try:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	88 start, end = fields
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	89 except ValueError:
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	90 raise RuntimeError("Unpacking problem", l)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	91 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	92 start = end = fields[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	93 start = int(start, 16)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	94 end = int(end, 16)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	95 for i in range(start, end+1):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	96 table[i] = i
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	97 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	98 code, value = fields
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	99 value = value.strip()
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	100 if value:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	101 value = [int(v, 16) for v in value.split(" ")]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	102 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	103 # table B.1
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	104 value = None
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	105 table[int(code, 16)] = value
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	106
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	107 ########### Generate compact Python versions of the tables #############
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	108
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	109 print("""# This file is generated by mkstringprep.py. DO NOT EDIT.
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	110 \"\"\"Library that exposes various tables found in the StringPrep RFC 3454.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	111
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	112 There are two kinds of tables: sets, for which a member test is provided,
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	113 and mappings, for which a mapping function is provided.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	114 \"\"\"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	115
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	116 import unicodedata
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	117 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	118
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	119 print("assert unicodedata.unidata_version == %s" % repr(unicodedata.unidata_version))
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	120
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	121 # A.1 is the table of unassigned characters
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	122 # XXX Plane 15 PUA is listed as unassigned in Python.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	123 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	124 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	125 assert name == "A.1"
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	126 table = set(table.keys())
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	127 Cn = set(gen_category(["Cn"]))
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	128
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	129 # FDD0..FDEF are process internal codes
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	130 Cn -= set(range(0xFDD0, 0xFDF0))
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	131 # not a character
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	132 Cn -= set(range(0xFFFE, 0x110000, 0x10000))
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	133 Cn -= set(range(0xFFFF, 0x110000, 0x10000))
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	134
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	135 # assert table == Cn
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	136
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	137 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	138 def in_table_a1(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	139 if unicodedata.category(code) != 'Cn': return False
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	140 c = ord(code)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	141 if 0xFDD0 <= c < 0xFDF0: return False
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	142 return (c & 0xFFFF) not in (0xFFFE, 0xFFFF)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	143 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	144
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	145 # B.1 cannot easily be derived
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	146 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	147 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	148 assert name == "B.1"
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	149 table = sorted(table.keys())
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	150 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	151 b1_set = """ + compact_set(table) + """
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	152 def in_table_b1(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	153 return ord(code) in b1_set
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	154 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	155
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	156 # B.2 and B.3 is case folding.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	157 # It takes CaseFolding.txt into account, which is
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	158 # not available in the Python database. Since
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	159 # B.2 is derived from B.3, we process B.3 first.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	160 # B.3 supposedly is CaseFolding-3.2.0.txt.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	161
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	162 name, table_b2 = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	163 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	164 assert name == "B.2"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	165
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	166 name, table_b3 = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	167 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	168 assert name == "B.3"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	169
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	170 # B.3 is mostly Python's .lower, except for a number
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	171 # of special cases, e.g. considering canonical forms.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	172
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	173 b3_exceptions = {}
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	174
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	175 for k,v in table_b2.items():
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	176 if map(ord, unichr(k).lower()) != v:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	177 b3_exceptions[k] = u"".join(map(unichr,v))
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	178
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	179 b3 = sorted(b3_exceptions.items())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	180
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	181 print("""
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	182 b3_exceptions = {""")
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	183 for i,(k,v) in enumerate(b3):
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	184 print("0x%x:%s," % (k, repr(v)), end=' ')
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	185 if i % 4 == 3:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	186 print()
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	187 print("}")
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	188
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	189 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	190 def map_table_b3(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	191 r = b3_exceptions.get(ord(code))
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	192 if r is not None: return r
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	193 return code.lower()
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	194 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	195
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	196 def map_table_b3(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	197 r = b3_exceptions.get(ord(code))
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	198 if r is not None: return r
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	199 return code.lower()
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	200
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	201 # B.2 is case folding for NFKC. This is the same as B.3,
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	202 # except where NormalizeWithKC(Fold(a)) !=
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	203 # NormalizeWithKC(Fold(NormalizeWithKC(Fold(a))))
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	204
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	205 def map_table_b2(a):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	206 al = map_table_b3(a)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	207 b = unicodedata.normalize("NFKC", al)
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	208 bl = "".join([map_table_b3(ch) for ch in b])
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	209 c = unicodedata.normalize("NFKC", bl)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	210 if b != c:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	211 return c
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	212 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	213 return al
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	214
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	215 specials = {}
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	216 for k,v in table_b2.items():
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	217 if list(map(ord, map_table_b2(chr(k)))) != v:
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	218 specials[k] = v
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	219
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	220 # B.3 should not add any additional special cases
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	221 assert specials == {}
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	222
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	223 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	224 def map_table_b2(a):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	225 al = map_table_b3(a)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	226 b = unicodedata.normalize("NFKC", al)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	227 bl = u"".join([map_table_b3(ch) for ch in b])
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	228 c = unicodedata.normalize("NFKC", bl)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	229 if b != c:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	230 return c
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	231 else:
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	232 return al
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	233 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	234
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	235 # C.1.1 is a table with a single character
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	236 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	237 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	238 assert name == "C.1.1"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	239 assert table == {0x20:0x20}
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	240
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	241 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	242 def in_table_c11(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	243 return code == u" "
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	244 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	245
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	246 # C.1.2 is the rest of all space characters
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	247 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	248 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	249 assert name == "C.1.2"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	250
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	251 # table = set(table.keys())
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	252 # Zs = set(gen_category(["Zs"])) - set([0x20])
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	253 # assert Zs == table
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	254
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	255 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	256 def in_table_c12(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	257 return unicodedata.category(code) == "Zs" and code != u" "
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	258
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	259 def in_table_c11_c12(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	260 return unicodedata.category(code) == "Zs"
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	261 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	262
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	263 # C.2.1 ASCII control characters
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	264 name, table_c21 = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	265 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	266 assert name == "C.2.1"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	267
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	268 Cc = set(gen_category(["Cc"]))
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	269 Cc_ascii = Cc & set(range(128))
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	270 table_c21 = set(table_c21.keys())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	271 assert Cc_ascii == table_c21
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	272
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	273 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	274 def in_table_c21(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	275 return ord(code) < 128 and unicodedata.category(code) == "Cc"
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	276 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	277
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	278 # C.2.2 Non-ASCII control characters. It also includes
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	279 # a number of characters in category Cf.
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	280 name, table_c22 = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	281 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	282 assert name == "C.2.2"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	283
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	284 Cc_nonascii = Cc - Cc_ascii
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	285 table_c22 = set(table_c22.keys())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	286 assert len(Cc_nonascii - table_c22) == 0
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	287
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	288 specials = list(table_c22 - Cc_nonascii)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	289 specials.sort()
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	290
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	291 print("""c22_specials = """ + compact_set(specials) + """
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	292 def in_table_c22(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	293 c = ord(code)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	294 if c < 128: return False
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	295 if unicodedata.category(code) == "Cc": return True
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	296 return c in c22_specials
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	297
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	298 def in_table_c21_c22(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	299 return unicodedata.category(code) == "Cc" or \\
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	300 ord(code) in c22_specials
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	301 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	302
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	303 # C.3 Private use
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	304 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	305 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	306 assert name == "C.3"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	307
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	308 Co = set(gen_category(["Co"]))
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	309 assert set(table.keys()) == Co
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	310
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	311 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	312 def in_table_c3(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	313 return unicodedata.category(code) == "Co"
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	314 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	315
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	316 # C.4 Non-character code points, xFFFE, xFFFF
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	317 # plus process internal codes
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	318 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	319 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	320 assert name == "C.4"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	321
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	322 nonchar = set(range(0xFDD0,0xFDF0))
63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	323 nonchar.update(range(0xFFFE,0x110000,0x10000))
63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	324 nonchar.update(range(0xFFFF,0x110000,0x10000))
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	325 table = set(table.keys())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	326 assert table == nonchar
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	327
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	328 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	329 def in_table_c4(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	330 c = ord(code)
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	331 if c < 0xFDD0: return False
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	332 if c < 0xFDF0: return True
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	333 return (ord(code) & 0xFFFF) in (0xFFFE, 0xFFFF)
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	334 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	335
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	336 # C.5 Surrogate codes
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	337 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	338 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	339 assert name == "C.5"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	340
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	341 Cs = set(gen_category(["Cs"]))
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	342 assert set(table.keys()) == Cs
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	343
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	344 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	345 def in_table_c5(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	346 return unicodedata.category(code) == "Cs"
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	347 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	348
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	349 # C.6 Inappropriate for plain text
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	350 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	351 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	352 assert name == "C.6"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	353
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	354 table = sorted(table.keys())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	355
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	356 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	357 c6_set = """ + compact_set(table) + """
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	358 def in_table_c6(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	359 return ord(code) in c6_set
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	360 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	361
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	362 # C.7 Inappropriate for canonical representation
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	363 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	364 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	365 assert name == "C.7"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	366
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	367 table = sorted(table.keys())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	368
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	369 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	370 c7_set = """ + compact_set(table) + """
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	371 def in_table_c7(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	372 return ord(code) in c7_set
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	373 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	374
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	375 # C.8 Change display properties or are deprecated
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	376 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	377 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	378 assert name == "C.8"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	379
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	380 table = sorted(table.keys())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	381
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	382 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	383 c8_set = """ + compact_set(table) + """
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	384 def in_table_c8(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	385 return ord(code) in c8_set
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	386 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	387
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	388 # C.9 Tagging characters
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	389 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	390 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	391 assert name == "C.9"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	392
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 42715 diff changeset	393 table = sorted(table.keys())
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	394
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	395 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	396 c9_set = """ + compact_set(table) + """
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	397 def in_table_c9(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	398 return ord(code) in c9_set
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	399 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	400
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	401 # D.1 Characters with bidirectional property "R" or "AL"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	402 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	403 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	404 assert name == "D.1"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	405
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	406 RandAL = set(gen_bidirectional(["R","AL"]))
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	407 assert set(table.keys()) == RandAL
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	408
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	409 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	410 def in_table_d1(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	411 return unicodedata.bidirectional(code) in ("R","AL")
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	412 """)
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	413
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	414 # D.2 Characters with bidirectional property "L"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	415 name, table = tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	416 del tables[0]
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	417 assert name == "D.2"
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	418
32125 b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	419 L = set(gen_bidirectional(["L"]))
b9edd3418a31 Applying SF patch #949329 on behalf of Raymond Hettinger. Armin Rigo <arigo@tunes.org> parents: 28671 diff changeset	420 assert set(table.keys()) == L
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	421
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	422 print("""
28671 d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	423 def in_table_d2(code):
d41d8de9768f Implement IDNA (Internationalized Domain Names in Applications). Martin v. Löwis <martin@v.loewis.de> parents: diff changeset	424 return unicodedata.bidirectional(code) == "L"
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 41642 diff changeset	425 """)

Mercurial > cpython-lru_cache-weakref

annotate Tools/unicode/mkstringprep.py @ 66978:aed2bbafa550 3.1