Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodectype.c @ 79675:c0a2294de787 2.7

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Issue #16199: Backport 2to3 typo fix from 3.2.
author Chris Jerdonek <chris.jerdonek@gmail.com>
date Thu, 11 Oct 2012 16:15:46 -0700
parents b36cb4602e21
children
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 /*
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
2 Unicode character type helpers.
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
3
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
4 Written by Marc-Andre Lemburg (mal@lemburg.com).
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
5 Modified for Python 2.0 by Fredrik Lundh (fredrik@pythonware.com)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
7 Copyright (c) Corporation for National Research Initiatives.
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
9 */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
10
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
11 #include "Python.h"
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
12 #include "unicodeobject.h"
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
13
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
14 #define ALPHA_MASK 0x01
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
15 #define DECIMAL_MASK 0x02
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
16 #define DIGIT_MASK 0x04
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
17 #define LOWER_MASK 0x08
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
18 #define LINEBREAK_MASK 0x10
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
19 #define SPACE_MASK 0x20
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
20 #define TITLE_MASK 0x40
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
21 #define UPPER_MASK 0x80
49124
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
22 #define NODELTA_MASK 0x100
56406
a9864417286b #1571184: makeunicodedata.py now generates the functions _PyUnicode_ToNumeric,
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents: 53450
diff changeset
23 #define NUMERIC_MASK 0x200
13380
d166a1e60760 Conditionally (currently on ifdef macintosh) break the large switch up
Jack Jansen <jack.jansen@cwi.nl>
parents: 13355
diff changeset
24
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
25 typedef struct {
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
26 const Py_UNICODE upper;
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
27 const Py_UNICODE lower;
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
28 const Py_UNICODE title;
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
29 const unsigned char decimal;
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
30 const unsigned char digit;
32197
3a84e23b042f - SF #962502: Add two more methods for unicode type; width() and
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31273
diff changeset
31 const unsigned short flags;
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
32 } _PyUnicode_TypeRecord;
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
33
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
34 #include "unicodetype_db.h"
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
35
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
36 static const _PyUnicode_TypeRecord *
18287
f6078a30f70a more unicode tweaks: fix unicodectype for sizeof(Py_UNICODE) >
Fredrik Lundh <fredrik@pythonware.com>
parents: 14854
diff changeset
37 gettyperecord(Py_UNICODE code)
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
38 {
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
39 int index;
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
40
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 25856
diff changeset
41 #ifdef Py_UNICODE_WIDE
25852
1f5f6481886d Update to Unicode 3.2 database.
Martin v. Löwis <martin@v.loewis.de>
parents: 18332
diff changeset
42 if (code >= 0x110000)
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
43 index = 0;
31273
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 25856
diff changeset
44 else
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 25856
diff changeset
45 #endif
25519d6c97b8 Fix gcc 3.3 warnings related to Py_UNICODE_WIDE.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 25856
diff changeset
46 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
47 index = index1[(code>>SHIFT)];
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
48 index = index2[(index<<SHIFT)+(code&((1<<SHIFT)-1))];
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
49 }
18287
f6078a30f70a more unicode tweaks: fix unicodectype for sizeof(Py_UNICODE) >
Fredrik Lundh <fredrik@pythonware.com>
parents: 14854
diff changeset
50
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
51 return &_PyUnicode_TypeRecords[index];
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
52 }
13380
d166a1e60760 Conditionally (currently on ifdef macintosh) break the large switch up
Jack Jansen <jack.jansen@cwi.nl>
parents: 13355
diff changeset
53
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 /* Returns the titlecase Unicode characters corresponding to ch or just
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 ch if no titlecase mapping is known. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
56
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18287
diff changeset
57 Py_UNICODE _PyUnicode_ToTitlecase(register Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
59 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
53450
c92e93a54298 Merged revisions 71947 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 49124
diff changeset
60 int delta = ctype->title;
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
61
49124
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
62 if (ctype->flags & NODELTA_MASK)
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
63 return delta;
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
64
25856
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
65 if (delta >= 32768)
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
66 delta -= 65536;
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
67
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
68 return ch + delta;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
69 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
70
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 /* Returns 1 for Unicode characters having the category 'Lt', 0
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
72 otherwise. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
73
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
74 int _PyUnicode_IsTitlecase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
76 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
77
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
78 return (ctype->flags & TITLE_MASK) != 0;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
80
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
81 /* Returns the integer decimal (0-9) for Unicode characters having
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
82 this property, -1 otherwise. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
83
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
84 int _PyUnicode_ToDecimalDigit(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
85 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
86 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
87
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
88 return (ctype->flags & DECIMAL_MASK) ? ctype->decimal : -1;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
89 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
90
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
91 int _PyUnicode_IsDecimalDigit(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
92 {
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
93 if (_PyUnicode_ToDecimalDigit(ch) < 0)
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
94 return 0;
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
95 return 1;
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
96 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
97
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
98 /* Returns the integer digit (0-9) for Unicode characters having
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
99 this property, -1 otherwise. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
100
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
101 int _PyUnicode_ToDigit(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
102 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
103 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
104
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
105 return (ctype->flags & DIGIT_MASK) ? ctype->digit : -1;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
106 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
107
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
108 int _PyUnicode_IsDigit(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
109 {
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
110 if (_PyUnicode_ToDigit(ch) < 0)
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
111 return 0;
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
112 return 1;
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
113 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
114
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
115 /* Returns the numeric value as double for Unicode characters having
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
116 this property, -1.0 otherwise. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
117
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
118 int _PyUnicode_IsNumeric(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
119 {
56406
a9864417286b #1571184: makeunicodedata.py now generates the functions _PyUnicode_ToNumeric,
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents: 53450
diff changeset
120 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
a9864417286b #1571184: makeunicodedata.py now generates the functions _PyUnicode_ToNumeric,
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents: 53450
diff changeset
121
a9864417286b #1571184: makeunicodedata.py now generates the functions _PyUnicode_ToNumeric,
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents: 53450
diff changeset
122 return (ctype->flags & NUMERIC_MASK) != 0;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
124
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
125 #ifndef WANT_WCTYPE_FUNCTIONS
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
126
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 /* Returns 1 for Unicode characters having the category 'Ll', 0
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
128 otherwise. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
129
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
130 int _PyUnicode_IsLowercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
131 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
132 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
133
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
134 return (ctype->flags & LOWER_MASK) != 0;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
135 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
136
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
137 /* Returns 1 for Unicode characters having the category 'Lu', 0
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
138 otherwise. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
139
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
140 int _PyUnicode_IsUppercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
141 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
142 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
143
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
144 return (ctype->flags & UPPER_MASK) != 0;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
145 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
146
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
147 /* Returns the uppercase Unicode characters corresponding to ch or just
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
148 ch if no uppercase mapping is known. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
149
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
150 Py_UNICODE _PyUnicode_ToUppercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
151 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
152 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
25856
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
153 int delta = ctype->upper;
49124
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
154 if (ctype->flags & NODELTA_MASK)
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
155 return delta;
25856
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
156 if (delta >= 32768)
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
157 delta -= 65536;
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
158 return ch + delta;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
159 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
160
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
161 /* Returns the lowercase Unicode characters corresponding to ch or just
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
162 ch if no lowercase mapping is known. */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
163
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
164 Py_UNICODE _PyUnicode_ToLowercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
165 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
166 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
25856
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
167 int delta = ctype->lower;
49124
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
168 if (ctype->flags & NODELTA_MASK)
c37c7dbd7880 Issue #3811: The Unicode database was updated to 5.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 38422
diff changeset
169 return delta;
25856
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
170 if (delta >= 32768)
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
171 delta -= 65536;
84a3dd994cd1 Make lower/upper/title work for non-BMP characters.
Martin v. Löwis <martin@v.loewis.de>
parents: 25852
diff changeset
172 return ch + delta;
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
173 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
174
13355
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
175 /* Returns 1 for Unicode characters having the category 'Ll', 'Lu', 'Lt',
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
176 'Lo' or 'Lm', 0 otherwise. */
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
177
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
178 int _PyUnicode_IsAlpha(Py_UNICODE ch)
13355
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
179 {
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
180 const _PyUnicode_TypeRecord *ctype = gettyperecord(ch);
13355
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
181
14854
89bc8475fc84 unicode database compression, step 3:
Fredrik Lundh <fredrik@pythonware.com>
parents: 13939
diff changeset
182 return (ctype->flags & ALPHA_MASK) != 0;
13355
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
183 }
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
184
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
185 #else
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
186
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
187 /* Export the interfaces using the wchar_t type for portability
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
188 reasons: */
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
189
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
190 int _PyUnicode_IsLowercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
191 {
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
192 return iswlower(ch);
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
193 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
194
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
195 int _PyUnicode_IsUppercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
196 {
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
197 return iswupper(ch);
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
198 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
199
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
200 Py_UNICODE _PyUnicode_ToLowercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
201 {
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
202 return towlower(ch);
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
203 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
204
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
205 Py_UNICODE _PyUnicode_ToUppercase(Py_UNICODE ch)
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
206 {
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
207 return towupper(ch);
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
208 }
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
209
18332
49acc8ac60c2 removed "register const" from scalar arguments to the unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 18324
diff changeset
210 int _PyUnicode_IsAlpha(Py_UNICODE ch)
13355
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
211 {
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
212 return iswalpha(ch);
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
213 }
b4aedd90c864 Added new lookup API which matches all alphabetic Unicode characters,
Marc-André Lemburg <mal@egenix.com>
parents: 12370
diff changeset
214
11967
507833b18213 Unicode character type helpers, written by Marc-Andre Lemburg.
Guido van Rossum <guido@python.org>
parents:
diff changeset
215 #endif