cpython-lru_cache-weakref: Tools/unicode/gencodec.py annotate

annotate Tools/unicode/gencodec.py @ 58689:514b56efd89a 3.1

Merged revisions 77962 via svnmerge from svn+ssh://pythondev@svn.python.org/python/branches/py3k ................ r77962 | r.david.murray | 2010-02-04 11:41:57 -0500 (Thu, 04 Feb 2010) | 17 lines Merged revisions 77943,77959-77960 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r77943 | r.david.murray | 2010-02-03 08:36:23 -0500 (Wed, 03 Feb 2010) | 2 lines Explicitly mention the default value for formatdate's usegmt parameter. ........ r77959 | nick.coghlan | 2010-02-04 07:43:58 -0500 (Thu, 04 Feb 2010) | 1 line Issue 6760: Clarify args handling for subprocess.Popen. Patch by Chris Rebert ........ r77960 | r.david.murray | 2010-02-04 11:33:31 -0500 (Thu, 04 Feb 2010) | 2 lines Add Chris Rebert to ACKS for issue 6760 Popen doc improvements. ........ ................

author	R. David Murray <rdmurray@bitdance.com>
date	Thu, 04 Feb 2010 16:44:31 +0000
parents	63741c7521aa
children	ef1b1198338b

rev	line source
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	1 """ Unicode Mapping Parser and Codec Generator.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	2
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	3 This script parses Unicode mapping files as available from the Unicode
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	4 site (ftp://ftp.unicode.org/Public/MAPPINGS/) and creates Python codec
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	5 modules from them. The codecs use the standard character mapping codec
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	6 to actually apply the mapping.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	7
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	8 Synopsis: gencodec.py dir codec_prefix
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	9
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	10 All files in dir are scanned and those producing non-empty mappings
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	11 will be written to <codec_prefix><mapname>.py with <mapname> being the
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	12 first part of the map's filename ('a' in a.b.c.txt) converted to
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	13 lowercase with hyphens replaced by underscores.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	14
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	15 The tool also writes marshalled versions of the mapping tables to the
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	16 same location (with .mapping extension).
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	17
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	18 Written by Marc-Andre Lemburg (mal@lemburg.com).
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	19
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	20 (c) Copyright CNRI, All Rights Reserved. NO WARRANTY.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	21 (c) Copyright Guido van Rossum, 2000.
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	22
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	23 Table generation:
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	24 (c) Copyright Marc-Andre Lemburg, 2005.
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	25 Licensed to PSF under a Contributor Agreement.
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	26
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	27 """#"
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	28
45471 7dd2c1746267 Merged revisions 60990-61002 via svnmerge from Christian Heimes <christian@cheimes.de> parents: 42427 diff changeset	29 import re, os, marshal, codecs
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	30
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	31 # Maximum allowed size of charmap tables
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	32 MAX_TABLE_SIZE = 8192
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	33
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	34 # Standard undefined Unicode code point
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	35 UNI_UNDEFINED = chr(0xFFFE)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	36
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	37 mapRE = re.compile('((?:0x[0-9a-fA-F]+\+?)+)'
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	38 '\s+'
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	39 '((?:(?:0x[0-9a-fA-Z]+\|<[A-Za-z]+>)\+?)*)'
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	40 '\s*'
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	41 '(#.+)?')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	42
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	43 def parsecodes(codes,
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	44 len=len, filter=filter,range=range):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	45
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	46 """ Converts code combinations to either a single code integer
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	47 or a tuple of integers.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	48
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	49 meta-codes (in angular brackets, e.g. <LR> and <RL>) are
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	50 ignored.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	51
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	52 Empty codes or illegal ones are returned as None.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	53
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	54 """
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	55 if not codes:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	56 return None
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	57 l = codes.split('+')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	58 if len(l) == 1:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	59 return int(l[0],16)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	60 for i in range(len(l)):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	61 try:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	62 l[i] = int(l[i],16)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	63 except ValueError:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	64 l[i] = None
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	65 l = [x for x in l if x is not None]
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	66 if len(l) == 1:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	67 return l[0]
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	68 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	69 return tuple(l)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	70
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	71 def readmap(filename):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	72
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	73 f = open(filename,'r')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	74 lines = f.readlines()
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	75 f.close()
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	76 enc2uni = {}
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	77 identity = []
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	78 unmapped = list(range(256))
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	79
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	80 # UTC mapping tables per convention don't include the identity
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	81 # mappings for code points 0x00 - 0x1F and 0x7F, unless these are
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	82 # explicitly mapped to different characters or undefined
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	83 for i in list(range(32)) + [127]:
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	84 identity.append(i)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	85 unmapped.remove(i)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	86 enc2uni[i] = (i, 'CONTROL CHARACTER')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	87
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	88 for line in lines:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	89 line = line.strip()
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	90 if not line or line[0] == '#':
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	91 continue
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	92 m = mapRE.match(line)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	93 if not m:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	94 #print '* not matched: %s' % repr(line)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	95 continue
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	96 enc,uni,comment = m.groups()
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	97 enc = parsecodes(enc)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	98 uni = parsecodes(uni)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	99 if comment is None:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	100 comment = ''
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	101 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	102 comment = comment[1:].strip()
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	103 if enc < 256:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	104 if enc in unmapped:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	105 unmapped.remove(enc)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	106 if enc == uni:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	107 identity.append(enc)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	108 enc2uni[enc] = (uni,comment)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	109 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	110 enc2uni[enc] = (uni,comment)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	111
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	112 # If there are more identity-mapped entries than unmapped entries,
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	113 # it pays to generate an identity dictionary first, and add explicit
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	114 # mappings to None for the rest
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	115 if len(identity) >= len(unmapped):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	116 for enc in unmapped:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	117 enc2uni[enc] = (None, "")
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	118 enc2uni['IDENTITY'] = 256
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	119
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	120 return enc2uni
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	121
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	122 def hexrepr(t, precision=4):
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	123
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	124 if t is None:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	125 return 'None'
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	126 try:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	127 len(t)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	128 except:
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	129 return '0x%0*X' % (precision, t)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	130 try:
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	131 return '(' + ', '.join(['0x%0*X' % (precision, item)
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	132 for item in t]) + ')'
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 38673 diff changeset	133 except TypeError as why:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	134 print('* failed to convert %r: %s' % (t, why))
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	135 raise
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	136
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	137 def python_mapdef_code(varname, map, comments=1, precisions=(2, 4)):
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	138
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	139 l = []
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	140 append = l.append
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	141 if "IDENTITY" in map:
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	142 append("%s = codecs.make_identity_dict(range(%d))" %
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	143 (varname, map["IDENTITY"]))
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	144 append("%s.update({" % varname)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	145 splits = 1
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	146 del map["IDENTITY"]
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	147 identity = 1
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	148 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	149 append("%s = {" % varname)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	150 splits = 0
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	151 identity = 0
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	152
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	153 mappings = sorted(map.items())
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	154 i = 0
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	155 key_precision, value_precision = precisions
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	156 for mapkey, mapvalue in mappings:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	157 mapcomment = ''
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	158 if isinstance(mapkey, tuple):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	159 (mapkey, mapcomment) = mapkey
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	160 if isinstance(mapvalue, tuple):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	161 (mapvalue, mapcomment) = mapvalue
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	162 if mapkey is None:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	163 continue
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	164 if (identity and
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	165 mapkey == mapvalue and
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	166 mapkey < 256):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	167 # No need to include identity mappings, since these
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	168 # are already set for the first 256 code points.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	169 continue
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	170 key = hexrepr(mapkey, key_precision)
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	171 value = hexrepr(mapvalue, value_precision)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	172 if mapcomment and comments:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	173 append(' %s: %s,\t# %s' % (key, value, mapcomment))
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	174 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	175 append(' %s: %s,' % (key, value))
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	176 i += 1
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	177 if i == 4096:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	178 # Split the definition into parts to that the Python
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	179 # parser doesn't dump core
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	180 if splits == 0:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	181 append('}')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	182 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	183 append('})')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	184 append('%s.update({' % varname)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	185 i = 0
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	186 splits = splits + 1
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	187 if splits == 0:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	188 append('}')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	189 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	190 append('})')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	191
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	192 return l
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	193
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	194 def python_tabledef_code(varname, map, comments=1, key_precision=2):
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	195
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	196 l = []
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	197 append = l.append
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	198 append('%s = (' % varname)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	199
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	200 # Analyze map and create table dict
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	201 mappings = sorted(map.items())
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	202 table = {}
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	203 maxkey = 0
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	204 if 'IDENTITY' in map:
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	205 for key in range(256):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	206 table[key] = (key, '')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	207 maxkey = 255
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	208 del map['IDENTITY']
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	209 for mapkey, mapvalue in mappings:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	210 mapcomment = ''
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	211 if isinstance(mapkey, tuple):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	212 (mapkey, mapcomment) = mapkey
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	213 if isinstance(mapvalue, tuple):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	214 (mapvalue, mapcomment) = mapvalue
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	215 if mapkey is None:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	216 continue
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	217 table[mapkey] = (mapvalue, mapcomment)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	218 if mapkey > maxkey:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	219 maxkey = mapkey
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	220 if maxkey > MAX_TABLE_SIZE:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	221 # Table too large
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	222 return None
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	223
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	224 # Create table code
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	225 for key in range(maxkey + 1):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	226 if key not in table:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	227 mapvalue = None
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	228 mapcomment = 'UNDEFINED'
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	229 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	230 mapvalue, mapcomment = table[key]
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	231 if mapvalue is None:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	232 mapchar = UNI_UNDEFINED
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	233 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	234 if isinstance(mapvalue, tuple):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	235 # 1-n mappings not supported
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	236 return None
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	237 else:
47103 63741c7521aa More 2to3 fixes in the Tools directory. Fixes #2893. Georg Brandl <georg@python.org> parents: 45471 diff changeset	238 mapchar = chr(mapvalue)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	239 if mapcomment and comments:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	240 append(' %r\t# %s -> %s' % (mapchar,
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	241 hexrepr(key, key_precision),
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	242 mapcomment))
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	243 else:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	244 append(' %r' % mapchar)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	245
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	246 append(')')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	247 return l
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	248
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	249 def codegen(name, map, encodingname, comments=1):
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	250
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	251 """ Returns Python source for the given map.
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	252
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	253 Comments are included in the source, if comments is true (default).
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	254
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	255 """
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	256 # Generate code
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	257 decoding_map_code = python_mapdef_code(
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	258 'decoding_map',
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	259 map,
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	260 comments=comments)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	261 decoding_table_code = python_tabledef_code(
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	262 'decoding_table',
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	263 map,
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	264 comments=comments)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	265 encoding_map_code = python_mapdef_code(
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	266 'encoding_map',
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	267 codecs.make_encoding_map(map),
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	268 comments=comments,
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	269 precisions=(4, 2))
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	270
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	271 if decoding_table_code:
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	272 suffix = 'table'
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	273 else:
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	274 suffix = 'map'
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	275
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	276 l = [
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	277 '''\
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	278 """ Python Character Mapping Codec %s generated from '%s' with gencodec.py.
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	279
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	280 """#"
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	281
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	282 import codecs
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	283
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	284 ### Codec APIs
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	285
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	286 class Codec(codecs.Codec):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	287
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	288 def encode(self,input,errors='strict'):
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	289 return codecs.charmap_encode(input,errors,encoding_%s)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	290
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	291 def decode(self,input,errors='strict'):
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	292 return codecs.charmap_decode(input,errors,decoding_%s)
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	293 ''' % (encodingname, name, suffix, suffix)]
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	294 l.append('''\
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	295 class IncrementalEncoder(codecs.IncrementalEncoder):
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	296 def encode(self, input, final=False):
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	297 return codecs.charmap_encode(input,self.errors,encoding_%s)[0]
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	298
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	299 class IncrementalDecoder(codecs.IncrementalDecoder):
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	300 def decode(self, input, final=False):
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	301 return codecs.charmap_decode(input,self.errors,decoding_%s)[0]''' %
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	302 (suffix, suffix))
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	303
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	304 l.append('''
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	305 class StreamWriter(Codec,codecs.StreamWriter):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	306 pass
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	307
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	308 class StreamReader(Codec,codecs.StreamReader):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	309 pass
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	310
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	311 ### encodings module API
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	312
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	313 def getregentry():
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	314 return codecs.CodecInfo(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	315 name=%r,
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	316 encode=Codec().encode,
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	317 decode=Codec().decode,
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	318 incrementalencoder=IncrementalEncoder,
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	319 incrementaldecoder=IncrementalDecoder,
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	320 streamreader=StreamReader,
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	321 streamwriter=StreamWriter,
38453 29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk. Thomas Wouters <thomas@python.org> parents: 37879 diff changeset	322 )
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	323 ''' % encodingname.replace('_', '-'))
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	324
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	325 # Add decoding table or map (with preference to the table)
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	326 if not decoding_table_code:
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	327 l.append('''
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	328 ### Decoding Map
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	329 ''')
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	330 l.extend(decoding_map_code)
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	331 else:
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	332 l.append('''
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	333 ### Decoding Table
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	334 ''')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	335 l.extend(decoding_table_code)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	336
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	337 # Add encoding map
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	338 if decoding_table_code:
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	339 l.append('''
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	340 ### Encoding table
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	341 encoding_table=codecs.charmap_build(decoding_table)
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	342 ''')
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	343 else:
a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	344 l.append('''
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	345 ### Encoding Map
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	346 ''')
38673 a19c20c5e5c7 Merge the rest of the trunk. Thomas Wouters <thomas@python.org> parents: 38453 diff changeset	347 l.extend(encoding_map_code)
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	348
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	349 # Final new-line
37879 f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair Thomas Wouters <thomas@python.org> parents: 37875 diff changeset	350 l.append('')
36121 686306a67e9b Whitespace normalization. Tim Peters <tim.peters@gmail.com> parents: 35800 diff changeset	351
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	352 return '\n'.join(l).expandtabs()
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	353
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	354 def pymap(name,map,pyfile,encodingname,comments=1):
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	355
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	356 code = codegen(name,map,encodingname,comments)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	357 f = open(pyfile,'w')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	358 f.write(code)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	359 f.close()
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	360
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	361 def marshalmap(name,map,marshalfile):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	362
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	363 d = {}
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	364 for e,(u,c) in map.items():
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	365 d[e] = (u,c)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	366 f = open(marshalfile,'wb')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	367 marshal.dump(d,f)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	368 f.close()
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	369
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	370 def convertdir(dir, dirprefix='', nameprefix='', comments=1):
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	371
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	372 mapnames = os.listdir(dir)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	373 for mapname in mapnames:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	374 mappathname = os.path.join(dir, mapname)
35800 4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	375 if not os.path.isfile(mappathname):
4bf6459c8e3b Apply some cosmetic fixes to the output of the script. Marc-André Lemburg <mal@egenix.com> parents: 35760 diff changeset	376 continue
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	377 name = os.path.split(mapname)[1]
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	378 name = name.replace('-','_')
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	379 name = name.split('.')[0]
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	380 name = name.lower()
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	381 name = nameprefix + name
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	382 codefile = name + '.py'
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	383 marshalfile = name + '.mapping'
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	384 print('converting %s to %s and %s' % (mapname,
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	385 dirprefix + codefile,
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	386 dirprefix + marshalfile))
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	387 try:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	388 map = readmap(os.path.join(dir,mapname))
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	389 if not map:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	390 print('* map is empty; skipping')
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	391 else:
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	392 pymap(mappathname, map, dirprefix + codefile,name,comments)
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	393 marshalmap(mappathname, map, dirprefix + marshalfile)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 38673 diff changeset	394 except ValueError as why:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	395 print('* conversion failed: %s' % why)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	396 raise
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	397
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	398 def rewritepythondir(dir, dirprefix='', comments=1):
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	399
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	400 mapnames = os.listdir(dir)
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	401 for mapname in mapnames:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	402 if not mapname.endswith('.mapping'):
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	403 continue
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	404 name = mapname[:-len('.mapping')]
a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	405 codefile = name + '.py'
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	406 print('converting %s to %s' % (mapname,
46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	407 dirprefix + codefile))
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	408 try:
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	409 map = marshal.load(open(os.path.join(dir,mapname),
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	410 'rb'))
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	411 if not map:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	412 print('* map is empty; skipping')
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	413 else:
37875 a4c70f3968be Merge part of the trunk changes into the p3yk branch. This merges from 43030 Thomas Wouters <thomas@python.org> parents: 37048 diff changeset	414 pymap(mapname, map, dirprefix + codefile,name,comments)
40587 6b18a095c2e7 SF patch 1631942 by Collin Winter: Guido van Rossum <guido@python.org> parents: 38673 diff changeset	415 except ValueError as why:
42427 46625e6122e9 Convert print statements to function calls in Tools/. Collin Winter <collinw@gmail.com> parents: 40587 diff changeset	416 print('* conversion failed: %s' % why)
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	417
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	418 if __name__ == '__main__':
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	419
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	420 import sys
79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	421 if 1:
37048 c048a09d06ff Remove apply() Neal Norwitz <nnorwitz@gmail.com> parents: 36121 diff changeset	422 convertdir(*sys.argv[1:])
35760 79c5b817fedc Moved gencodec.py to the Tools/unicode/ directory. Marc-André Lemburg <mal@egenix.com> parents: diff changeset	423 else:
37048 c048a09d06ff Remove apply() Neal Norwitz <nnorwitz@gmail.com> parents: 36121 diff changeset	424 rewritepythondir(*sys.argv[1:])

Mercurial > cpython-lru_cache-weakref

annotate Tools/unicode/gencodec.py @ 58689:514b56efd89a 3.1