Mercurial > cpython-lru_cache-weakref
annotate Include/unicodeobject.h @ 83447:678564fc0962 3.3
#17789: test_random now works with unittest test discovery. Patch by Zachary Ware.
| author | Ezio Melotti <ezio.melotti@gmail.com> |
|---|---|
| date | Fri, 19 Apr 2013 05:45:27 +0300 |
| parents | a1aa13ef00c5 |
| children | 89ee959a9b54 |
| rev | line source |
|---|---|
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1 #ifndef Py_UNICODEOBJECT_H |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2 #define Py_UNICODEOBJECT_H |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
3 |
|
44999
7c02182e2979
Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44586
diff
changeset
|
4 #include <stdarg.h> |
|
7c02182e2979
Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44586
diff
changeset
|
5 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
6 /* |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
7 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
8 Unicode implementation based on original code by Fredrik Lundh, |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
9 modified by Marc-Andre Lemburg (mal@lemburg.com) according to the |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
10 Unicode Integration Proposal. (See |
|
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
11 http://www.egenix.com/files/python/unicode-proposal.txt). |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
12 |
|
13828
254f8b7c4a83
Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents:
13661
diff
changeset
|
13 Copyright (c) Corporation for National Research Initiatives. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
14 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
15 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
16 Original header: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
17 -------------------------------------------------------------------- |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
18 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
19 * Yet another Unicode string type for Python. This type supports the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
20 * 16-bit Basic Multilingual Plane (BMP) only. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
21 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
22 * Written by Fredrik Lundh, January 1999. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
23 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
24 * Copyright (c) 1999 by Secret Labs AB. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
25 * Copyright (c) 1999 by Fredrik Lundh. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
26 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
27 * fredrik@pythonware.com |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
28 * http://www.pythonware.com |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
29 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
30 * -------------------------------------------------------------------- |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
31 * This Unicode String Type is |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
32 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
33 * Copyright (c) 1999 by Secret Labs AB |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
34 * Copyright (c) 1999 by Fredrik Lundh |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
35 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
36 * By obtaining, using, and/or copying this software and/or its |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
37 * associated documentation, you agree that you have read, understood, |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
38 * and will comply with the following terms and conditions: |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
39 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
40 * Permission to use, copy, modify, and distribute this software and its |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
41 * associated documentation for any purpose and without fee is hereby |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
42 * granted, provided that the above copyright notice appears in all |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
43 * copies, and that both that copyright notice and this permission notice |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
44 * appear in supporting documentation, and that the name of Secret Labs |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
45 * AB or the author not be used in advertising or publicity pertaining to |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
46 * distribution of the software without specific, written prior |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
47 * permission. |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
48 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
49 * SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
50 * THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
51 * FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
52 * ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
53 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
54 * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
55 * OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
56 * -------------------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
57 |
| 19980 | 58 #include <ctype.h> |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
59 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
60 /* === Internal API ======================================================= */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
61 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
62 /* --- Internal Unicode Format -------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
63 |
|
44586
a2938c55200c
Always define Py_USING_UNICODE, 3rd party software may depend on it. A missing declaration can lead to strange bugs as I had to learn the hard way in the upcoming merge
Christian Heimes <christian@cheimes.de>
parents:
44351
diff
changeset
|
64 /* Python 3.x requires unicode */ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
65 #define Py_USING_UNICODE |
|
44586
a2938c55200c
Always define Py_USING_UNICODE, 3rd party software may depend on it. A missing declaration can lead to strange bugs as I had to learn the hard way in the upcoming merge
Christian Heimes <christian@cheimes.de>
parents:
44351
diff
changeset
|
66 |
| 72475 | 67 #ifndef SIZEOF_WCHAR_T |
| 68 #error Must define SIZEOF_WCHAR_T | |
|
18296
22f85c6a6520
Make Unicode work a bit better on Windows...
Fredrik Lundh <fredrik@pythonware.com>
parents:
18292
diff
changeset
|
69 #endif |
|
22f85c6a6520
Make Unicode work a bit better on Windows...
Fredrik Lundh <fredrik@pythonware.com>
parents:
18292
diff
changeset
|
70 |
| 72475 | 71 #define Py_UNICODE_SIZE SIZEOF_WCHAR_T |
| 72 | |
| 73 /* If wchar_t can be used for UCS-4 storage, set Py_UNICODE_WIDE. | |
| 74 Otherwise, Unicode strings are stored as UCS-2 (with limited support | |
| 75 for UTF-16) */ | |
|
18324
3ba7a358ce02
use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents:
18298
diff
changeset
|
76 |
|
3ba7a358ce02
use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents:
18298
diff
changeset
|
77 #if Py_UNICODE_SIZE >= 4 |
|
3ba7a358ce02
use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents:
18298
diff
changeset
|
78 #define Py_UNICODE_WIDE |
|
18292
ffbb9c918ac0
Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents:
18280
diff
changeset
|
79 #endif |
|
18280
0632dcd69ea3
experimental UCS-4 support: added USE_UCS4_STORAGE define to
Fredrik Lundh <fredrik@pythonware.com>
parents:
17887
diff
changeset
|
80 |
|
64687
9cdba6864eb6
#9210: remove --with-wctype-functions configure option.
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64639
diff
changeset
|
81 /* Set these flags if the platform has "wchar.h" and the |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
82 wchar_t type is a 16-bit unsigned type */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
83 /* #define HAVE_WCHAR_H */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
84 /* #define HAVE_USABLE_WCHAR_T */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
85 |
| 72475 | 86 /* Py_UNICODE was the native Unicode storage format (code unit) used by |
| 87 Python and represents a single Unicode element in the Unicode type. | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
88 With PEP 393, Py_UNICODE is deprecated and replaced with a |
| 72475 | 89 typedef to wchar_t. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
90 |
| 72475 | 91 #ifndef Py_LIMITED_API |
| 92 #define PY_UNICODE_TYPE wchar_t | |
| 93 typedef wchar_t Py_UNICODE; | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
94 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
95 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
96 /* If the compiler provides a wchar_t type we try to support it |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
97 through the interface functions PyUnicode_FromWideChar(), |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
98 PyUnicode_AsWideChar() and PyUnicode_AsWideCharString(). */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
99 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
100 #ifdef HAVE_USABLE_WCHAR_T |
|
13917
03fa40528d17
Added #fndef's to avoid compiler errors.
Marc-André Lemburg <mal@egenix.com>
parents:
13837
diff
changeset
|
101 # ifndef HAVE_WCHAR_H |
|
03fa40528d17
Added #fndef's to avoid compiler errors.
Marc-André Lemburg <mal@egenix.com>
parents:
13837
diff
changeset
|
102 # define HAVE_WCHAR_H |
|
03fa40528d17
Added #fndef's to avoid compiler errors.
Marc-André Lemburg <mal@egenix.com>
parents:
13837
diff
changeset
|
103 # endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
104 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
105 |
| 72475 | 106 #if defined(MS_WINDOWS) |
|
71193
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
107 # define HAVE_MBCS |
|
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
108 #endif |
|
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
109 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
110 #ifdef HAVE_WCHAR_H |
| 12074 | 111 /* Work around a cosmetic bug in BSDI 4.x wchar.h; thanks to Thomas Wouters */ |
| 112 # ifdef _HAVE_BSDI | |
| 113 # include <time.h> | |
| 114 # endif | |
| 19980 | 115 # include <wchar.h> |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
116 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
117 |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
118 /* Py_UCS4 and Py_UCS2 are typedefs for the respective |
| 72475 | 119 unicode representations. */ |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
120 #if SIZEOF_INT == 4 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
121 typedef unsigned int Py_UCS4; |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
122 #elif SIZEOF_LONG == 4 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
123 typedef unsigned long Py_UCS4; |
| 72475 | 124 #else |
| 125 #error "Could not find a proper typedef for Py_UCS4" | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
126 #endif |
|
13393
bcb14627b8f4
Bill Tutt: Added Py_UCS4 typedef to hold UCS4 values (these need
Marc-André Lemburg <mal@egenix.com>
parents:
13352
diff
changeset
|
127 |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
128 #if SIZEOF_SHORT == 2 |
| 72475 | 129 typedef unsigned short Py_UCS2; |
|
18911
55b45b47fc48
As discussed on python-dev: this patch adds name mangling to
Marc-André Lemburg <mal@egenix.com>
parents:
18887
diff
changeset
|
130 #else |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
131 #error "Could not find a proper typedef for Py_UCS2" |
|
18911
55b45b47fc48
As discussed on python-dev: this patch adds name mangling to
Marc-André Lemburg <mal@egenix.com>
parents:
18887
diff
changeset
|
132 #endif |
|
55b45b47fc48
As discussed on python-dev: this patch adds name mangling to
Marc-André Lemburg <mal@egenix.com>
parents:
18887
diff
changeset
|
133 |
| 72475 | 134 typedef unsigned char Py_UCS1; |
|
18911
55b45b47fc48
As discussed on python-dev: this patch adds name mangling to
Marc-André Lemburg <mal@egenix.com>
parents:
18887
diff
changeset
|
135 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
136 /* --- Internal Unicode Operations ---------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
137 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
138 /* Since splitting on whitespace is an important use case, and |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
139 whitespace in most situations is solely ASCII whitespace, we |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
140 optimize for the common case by using a quick look-up table |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
141 _Py_ascii_whitespace (see below) with an inlined check. |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
142 |
|
45056
c8fca9b5cd3c
Merged revisions 60408-60440 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44999
diff
changeset
|
143 */ |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
144 #ifndef Py_LIMITED_API |
|
45056
c8fca9b5cd3c
Merged revisions 60408-60440 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44999
diff
changeset
|
145 #define Py_UNICODE_ISSPACE(ch) \ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
146 ((ch) < 128U ? _Py_ascii_whitespace[(ch)] : _PyUnicode_IsWhitespace(ch)) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
147 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
148 #define Py_UNICODE_ISLOWER(ch) _PyUnicode_IsLowercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
149 #define Py_UNICODE_ISUPPER(ch) _PyUnicode_IsUppercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
150 #define Py_UNICODE_ISTITLE(ch) _PyUnicode_IsTitlecase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
151 #define Py_UNICODE_ISLINEBREAK(ch) _PyUnicode_IsLinebreak(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
152 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
153 #define Py_UNICODE_TOLOWER(ch) _PyUnicode_ToLowercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
154 #define Py_UNICODE_TOUPPER(ch) _PyUnicode_ToUppercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
155 #define Py_UNICODE_TOTITLE(ch) _PyUnicode_ToTitlecase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
156 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
157 #define Py_UNICODE_ISDECIMAL(ch) _PyUnicode_IsDecimalDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
158 #define Py_UNICODE_ISDIGIT(ch) _PyUnicode_IsDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
159 #define Py_UNICODE_ISNUMERIC(ch) _PyUnicode_IsNumeric(ch) |
|
47611
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
160 #define Py_UNICODE_ISPRINTABLE(ch) _PyUnicode_IsPrintable(ch) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
161 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
162 #define Py_UNICODE_TODECIMAL(ch) _PyUnicode_ToDecimalDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
163 #define Py_UNICODE_TODIGIT(ch) _PyUnicode_ToDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
164 #define Py_UNICODE_TONUMERIC(ch) _PyUnicode_ToNumeric(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
165 |
|
13352
d88535ccaee5
Modified the ISALPHA and ISALNUM macros to use the new lookup APIs
Marc-André Lemburg <mal@egenix.com>
parents:
13320
diff
changeset
|
166 #define Py_UNICODE_ISALPHA(ch) _PyUnicode_IsAlpha(ch) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
167 |
|
13320
67a04816ea50
Added new Py_UNICODE_ISALPHA() and Py_UNICODE_ISALNUM() macros
Marc-André Lemburg <mal@egenix.com>
parents:
12956
diff
changeset
|
168 #define Py_UNICODE_ISALNUM(ch) \ |
|
67a04816ea50
Added new Py_UNICODE_ISALPHA() and Py_UNICODE_ISALNUM() macros
Marc-André Lemburg <mal@egenix.com>
parents:
12956
diff
changeset
|
169 (Py_UNICODE_ISALPHA(ch) || \ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
170 Py_UNICODE_ISDECIMAL(ch) || \ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
171 Py_UNICODE_ISDIGIT(ch) || \ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
172 Py_UNICODE_ISNUMERIC(ch)) |
|
13320
67a04816ea50
Added new Py_UNICODE_ISALPHA() and Py_UNICODE_ISALNUM() macros
Marc-André Lemburg <mal@egenix.com>
parents:
12956
diff
changeset
|
173 |
| 72475 | 174 #define Py_UNICODE_COPY(target, source, length) \ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
175 Py_MEMCPY((target), (source), (length)*sizeof(Py_UNICODE)) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
176 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
177 #define Py_UNICODE_FILL(target, value, length) \ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
178 do {Py_ssize_t i_; Py_UNICODE *t_ = (target); Py_UNICODE v_ = (value);\ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
179 for (i_ = 0; i_ < (length); i_++) t_[i_] = v_;\ |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
180 } while (0) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
181 |
|
72036
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
182 /* macros to work with surrogates */ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
183 #define Py_UNICODE_IS_SURROGATE(ch) (0xD800 <= ch && ch <= 0xDFFF) |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
184 #define Py_UNICODE_IS_HIGH_SURROGATE(ch) (0xD800 <= ch && ch <= 0xDBFF) |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
185 #define Py_UNICODE_IS_LOW_SURROGATE(ch) (0xDC00 <= ch && ch <= 0xDFFF) |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
186 /* Join two surrogate characters and return a single Py_UCS4 value. */ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
187 #define Py_UNICODE_JOIN_SURROGATES(high, low) \ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
188 (((((Py_UCS4)(high) & 0x03FF) << 10) | \ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
189 ((Py_UCS4)(low) & 0x03FF)) + 0x10000) |
|
73793
351cdf87bed0
Py_UNICODE_HIGH_SURROGATE() and Py_UNICODE_LOW_SURROGATE() macros
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73654
diff
changeset
|
190 /* high surrogate = top 10 bits added to D800 */ |
|
77448
acca141fda80
Issue #15026: utf-16 encoding is now significantly faster (up to 10x).
Antoine Pitrou <solipsis@pitrou.net>
parents:
77351
diff
changeset
|
191 #define Py_UNICODE_HIGH_SURROGATE(ch) (0xD800 - (0x10000 >> 10) + ((ch) >> 10)) |
|
73793
351cdf87bed0
Py_UNICODE_HIGH_SURROGATE() and Py_UNICODE_LOW_SURROGATE() macros
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73654
diff
changeset
|
192 /* low surrogate = bottom 10 bits added to DC00 */ |
|
77448
acca141fda80
Issue #15026: utf-16 encoding is now significantly faster (up to 10x).
Antoine Pitrou <solipsis@pitrou.net>
parents:
77351
diff
changeset
|
193 #define Py_UNICODE_LOW_SURROGATE(ch) (0xDC00 + ((ch) & 0x3FF)) |
|
72036
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
194 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
195 /* Check if substring matches at given offset. The offset must be |
|
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
196 valid, and the substring must not be empty. */ |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
197 |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
198 #define Py_UNICODE_MATCH(string, offset, substring) \ |
| 72475 | 199 ((*((string)->wstr + (offset)) == *((substring)->wstr)) && \ |
| 200 ((*((string)->wstr + (offset) + (substring)->wstr_length-1) == *((substring)->wstr + (substring)->wstr_length-1))) && \ | |
| 201 !memcmp((string)->wstr + (offset), (substring)->wstr, (substring)->wstr_length*sizeof(Py_UNICODE))) | |
| 202 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
203 #endif /* Py_LIMITED_API */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
204 |
|
12025
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
205 #ifdef __cplusplus |
|
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
206 extern "C" { |
|
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
207 #endif |
|
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
208 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
209 /* --- Unicode Type ------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
210 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
211 #ifndef Py_LIMITED_API |
| 72475 | 212 |
| 213 /* ASCII-only strings created through PyUnicode_New use the PyASCIIObject | |
| 214 structure. state.ascii and state.compact are set, and the data | |
| 215 immediately follow the structure. utf8_length and wstr_length can be found | |
| 216 in the length field; the utf8 pointer is equal to the data pointer. */ | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
217 typedef struct { |
| 72729 | 218 /* There are 4 forms of Unicode strings: |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
219 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
220 - compact ascii: |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
221 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
222 * structure = PyASCIIObject |
|
73941
0eea55925d1e
resize_copy() now supports legacy ready strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73939
diff
changeset
|
223 * test: PyUnicode_IS_COMPACT_ASCII(op) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
224 * kind = PyUnicode_1BYTE_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
225 * compact = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
226 * ascii = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
227 * ready = 1 |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
228 * (length is the length of the utf8 and wstr strings) |
|
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
229 * (data starts just after the structure) |
|
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
230 * (since ASCII is decoded from UTF-8, the utf8 string are the data) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
231 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
232 - compact: |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
233 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
234 * structure = PyCompactUnicodeObject |
|
74124
4bbc3ffa1207
fix PyCompactUnicodeObject doc (test)
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74087
diff
changeset
|
235 * test: PyUnicode_IS_COMPACT(op) && !PyUnicode_IS_ASCII(op) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
236 * kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
237 PyUnicode_4BYTE_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
238 * compact = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
239 * ready = 1 |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
240 * ascii = 0 |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
241 * utf8 is not shared with data |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
242 * utf8_length = 0 if utf8 is NULL |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
243 * wstr is shared with data and wstr_length=length |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
244 if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2 |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
245 or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_t)=4 |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
246 * wstr_length = 0 if wstr is NULL |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
247 * (data starts just after the structure) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
248 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
249 - legacy string, not ready: |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
250 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
251 * structure = PyUnicodeObject |
|
73941
0eea55925d1e
resize_copy() now supports legacy ready strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73939
diff
changeset
|
252 * test: kind == PyUnicode_WCHAR_KIND |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
253 * length = 0 (use wstr_length) |
|
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
254 * hash = -1 |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
255 * kind = PyUnicode_WCHAR_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
256 * compact = 0 |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
257 * ascii = 0 |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
258 * ready = 0 |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
259 * interned = SSTATE_NOT_INTERNED |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
260 * wstr is not NULL |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
261 * data.any is NULL |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
262 * utf8 is NULL |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
263 * utf8_length = 0 |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
264 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
265 - legacy string, ready: |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
266 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
267 * structure = PyUnicodeObject structure |
|
73941
0eea55925d1e
resize_copy() now supports legacy ready strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73939
diff
changeset
|
268 * test: !PyUnicode_IS_COMPACT(op) && kind != PyUnicode_WCHAR_KIND |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
269 * kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
270 PyUnicode_4BYTE_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
271 * compact = 0 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
272 * ready = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
273 * data.any is not NULL |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
274 * utf8 is shared and utf8_length = length with data.any if ascii = 1 |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
275 * utf8_length = 0 if utf8 is NULL |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
276 * wstr is shared with data.any and wstr_length = length |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
277 if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2 |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
278 or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_4)=4 |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
279 * wstr_length = 0 if wstr is NULL |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
280 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
281 Compact strings use only one memory block (structure + characters), |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
282 whereas legacy strings use one block for the structure and one block |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
283 for characters. |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
284 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
285 Legacy strings are created by PyUnicode_FromUnicode() and |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
286 PyUnicode_FromStringAndSize(NULL, size) functions. They become ready |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
287 when PyUnicode_READY() is called. |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
288 |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
289 See also _PyUnicode_CheckConsistency(). |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
290 */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
291 PyObject_HEAD |
| 72475 | 292 Py_ssize_t length; /* Number of code points in the string */ |
|
65494
e8969fb822fc
make hashes always the size of pointers; introduce Py_hash_t #9778
Benjamin Peterson <benjamin@python.org>
parents:
65348
diff
changeset
|
293 Py_hash_t hash; /* Hash value; -1 if not set */ |
| 72475 | 294 struct { |
| 295 /* | |
| 296 SSTATE_NOT_INTERNED (0) | |
| 297 SSTATE_INTERNED_MORTAL (1) | |
| 298 SSTATE_INTERNED_IMMORTAL (2) | |
| 299 | |
| 300 If interned != SSTATE_NOT_INTERNED, the two references from the | |
| 301 dictionary to this object are *not* counted in ob_refcnt. | |
| 302 */ | |
| 303 unsigned int interned:2; | |
| 304 /* Character size: | |
| 305 | |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
306 - PyUnicode_WCHAR_KIND (0): |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
307 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
308 * character type = wchar_t (16 or 32 bits, depending on the |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
309 platform) |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
310 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
311 - PyUnicode_1BYTE_KIND (1): |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
312 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
313 * character type = Py_UCS1 (8 bits, unsigned) |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
314 * all characters are in the range U+0000-U+00FF (latin1) |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
315 * if ascii is set, all characters are in the range U+0000-U+007F |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
316 (ASCII), otherwise at least one character is in the range |
|
72715
341c3002ffb2
rephrase PyUnicode_1BYTE_KIND documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72714
diff
changeset
|
317 U+0080-U+00FF |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
318 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
319 - PyUnicode_2BYTE_KIND (2): |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
320 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
321 * character type = Py_UCS2 (16 bits, unsigned) |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
322 * all characters are in the range U+0000-U+FFFF (BMP) |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
323 * at least one character is in the range U+0100-U+FFFF |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
324 |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
325 - PyUnicode_4BYTE_KIND (4): |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
326 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
327 * character type = Py_UCS4 (32 bits, unsigned) |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
328 * all characters are in the range U+0000-U+10FFFF |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
329 * at least one character is in the range U+10000-U+10FFFF |
| 72475 | 330 */ |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
331 unsigned int kind:3; |
| 72475 | 332 /* Compact is with respect to the allocation scheme. Compact unicode |
| 333 objects only require one memory block while non-compact objects use | |
| 334 one block for the PyUnicodeObject struct and another for its data | |
| 335 buffer. */ | |
| 336 unsigned int compact:1; | |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
337 /* The string only contains characters in the range U+0000-U+007F (ASCII) |
|
72715
341c3002ffb2
rephrase PyUnicode_1BYTE_KIND documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72714
diff
changeset
|
338 and the kind is PyUnicode_1BYTE_KIND. If ascii is set and compact is |
|
341c3002ffb2
rephrase PyUnicode_1BYTE_KIND documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72714
diff
changeset
|
339 set, use the PyASCIIObject structure. */ |
| 72475 | 340 unsigned int ascii:1; |
| 341 /* The ready flag indicates whether the object layout is initialized | |
| 342 completely. This means that this is either a compact object, or | |
| 343 the data pointer is filled out. The bit is redundant, and helps | |
| 344 to minimize the test in PyUnicode_IS_READY(). */ | |
| 345 unsigned int ready:1; | |
| 346 } state; | |
| 347 wchar_t *wstr; /* wchar_t representation (null-terminated) */ | |
| 348 } PyASCIIObject; | |
| 349 | |
| 350 /* Non-ASCII strings allocated through PyUnicode_New use the | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
351 PyCompactUnicodeObject structure. state.compact is set, and the data |
| 72475 | 352 immediately follow the structure. */ |
| 353 typedef struct { | |
| 354 PyASCIIObject _base; | |
| 355 Py_ssize_t utf8_length; /* Number of bytes in utf8, excluding the | |
| 356 * terminating \0. */ | |
| 357 char *utf8; /* UTF-8 representation (null-terminated) */ | |
| 358 Py_ssize_t wstr_length; /* Number of code points in wstr, possible | |
| 359 * surrogates count as two code points. */ | |
| 360 } PyCompactUnicodeObject; | |
| 361 | |
| 362 /* Strings allocated through PyUnicode_FromUnicode(NULL, len) use the | |
| 363 PyUnicodeObject structure. The actual string data is initially in the wstr | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
364 block, and copied into the data block using _PyUnicode_Ready. */ |
| 72475 | 365 typedef struct { |
| 366 PyCompactUnicodeObject _base; | |
| 367 union { | |
| 368 void *any; | |
| 369 Py_UCS1 *latin1; | |
| 370 Py_UCS2 *ucs2; | |
| 371 Py_UCS4 *ucs4; | |
| 372 } data; /* Canonical, smallest-form Unicode buffer */ | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
373 } PyUnicodeObject; |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
374 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
375 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
376 PyAPI_DATA(PyTypeObject) PyUnicode_Type; |
|
44061
0cc552ab76bd
Added all PyTypeObjects to the appropriate header files.
Christian Heimes <christian@cheimes.de>
parents:
43933
diff
changeset
|
377 PyAPI_DATA(PyTypeObject) PyUnicodeIter_Type; |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
378 |
|
40909
f2f0b41f270d
Merged revisions 53875-53911 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
39610
diff
changeset
|
379 #define PyUnicode_Check(op) \ |
|
44351
86a2679dca46
#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT.
Christian Heimes <christian@cheimes.de>
parents:
44061
diff
changeset
|
380 PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_UNICODE_SUBCLASS) |
|
86a2679dca46
#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT.
Christian Heimes <christian@cheimes.de>
parents:
44061
diff
changeset
|
381 #define PyUnicode_CheckExact(op) (Py_TYPE(op) == &PyUnicode_Type) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
382 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
383 /* Fast access macros */ |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
384 #ifndef Py_LIMITED_API |
| 72475 | 385 |
| 386 #define PyUnicode_WSTR_LENGTH(op) \ | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
387 (PyUnicode_IS_COMPACT_ASCII(op) ? \ |
| 72475 | 388 ((PyASCIIObject*)op)->length : \ |
| 389 ((PyCompactUnicodeObject*)op)->wstr_length) | |
| 390 | |
| 391 /* Returns the deprecated Py_UNICODE representation's size in code units | |
| 392 (this includes surrogate pairs as 2 units). | |
| 393 If the Py_UNICODE representation is not available, it will be computed | |
| 394 on request. Use PyUnicode_GET_LENGTH() for the length in code points. */ | |
| 395 | |
|
73654
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
396 #define PyUnicode_GET_SIZE(op) \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
397 (assert(PyUnicode_Check(op)), \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
398 (((PyASCIIObject *)(op))->wstr) ? \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
399 PyUnicode_WSTR_LENGTH(op) : \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
400 ((void)PyUnicode_AsUnicode((PyObject *)(op)), \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
401 assert(((PyASCIIObject *)(op))->wstr), \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
402 PyUnicode_WSTR_LENGTH(op))) |
| 72475 | 403 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
404 #define PyUnicode_GET_DATA_SIZE(op) \ |
| 72475 | 405 (PyUnicode_GET_SIZE(op) * Py_UNICODE_SIZE) |
| 406 | |
| 407 /* Alias for PyUnicode_AsUnicode(). This will create a wchar_t/Py_UNICODE | |
| 408 representation on demand. Using this macro is very inefficient now, | |
| 409 try to port your code to use the new PyUnicode_*BYTE_DATA() macros or | |
| 410 use PyUnicode_WRITE() and PyUnicode_READ(). */ | |
| 411 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
412 #define PyUnicode_AS_UNICODE(op) \ |
| 72475 | 413 (assert(PyUnicode_Check(op)), \ |
| 414 (((PyASCIIObject *)(op))->wstr) ? (((PyASCIIObject *)(op))->wstr) : \ | |
| 415 PyUnicode_AsUnicode((PyObject *)(op))) | |
| 416 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
417 #define PyUnicode_AS_DATA(op) \ |
| 72475 | 418 ((const char *)(PyUnicode_AS_UNICODE(op))) |
| 419 | |
| 420 | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
421 /* --- Flexible String Representation Helper Macros (PEP 393) -------------- */ |
| 72475 | 422 |
|
73591
abe62c9d659d
Fix misused of "PyUnicodeObject" structure name in unicodeobject.h
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73476
diff
changeset
|
423 /* Values for PyASCIIObject.state: */ |
| 72475 | 424 |
| 425 /* Interning state. */ | |
| 426 #define SSTATE_NOT_INTERNED 0 | |
| 427 #define SSTATE_INTERNED_MORTAL 1 | |
| 428 #define SSTATE_INTERNED_IMMORTAL 2 | |
| 429 | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
430 /* Return true if the string contains only ASCII characters, or 0 if not. The |
|
73939
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
431 string may be compact (PyUnicode_IS_COMPACT_ASCII) or not, but must be |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
432 ready. */ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
433 #define PyUnicode_IS_ASCII(op) \ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
434 (assert(PyUnicode_Check(op)), \ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
435 assert(PyUnicode_IS_READY(op)), \ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
436 ((PyASCIIObject*)op)->state.ascii) |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
437 |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
438 /* Return true if the string is compact or 0 if not. |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
439 No type checks or Ready calls are performed. */ |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
440 #define PyUnicode_IS_COMPACT(op) \ |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
441 (((PyASCIIObject*)(op))->state.compact) |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
442 |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
443 /* Return true if the string is a compact ASCII string (use PyASCIIObject |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
444 structure), or 0 if not. No type checks or Ready calls are performed. */ |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
445 #define PyUnicode_IS_COMPACT_ASCII(op) \ |
|
73939
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
446 (((PyASCIIObject*)op)->state.ascii && PyUnicode_IS_COMPACT(op)) |
| 72475 | 447 |
|
74087
d2df3396fef0
backout 7876cd49300d: Move PyUnicode_WCHAR_KIND outside PyUnicode_Kind enum
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74014
diff
changeset
|
448 enum PyUnicode_Kind { |
| 72475 | 449 /* String contains only wstr byte characters. This is only possible |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
450 when the string was created with a legacy API and _PyUnicode_Ready() |
| 72475 | 451 has not been called yet. */ |
|
74087
d2df3396fef0
backout 7876cd49300d: Move PyUnicode_WCHAR_KIND outside PyUnicode_Kind enum
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74014
diff
changeset
|
452 PyUnicode_WCHAR_KIND = 0, |
|
d2df3396fef0
backout 7876cd49300d: Move PyUnicode_WCHAR_KIND outside PyUnicode_Kind enum
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74014
diff
changeset
|
453 /* Return values of the PyUnicode_KIND() macro: */ |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
454 PyUnicode_1BYTE_KIND = 1, |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
455 PyUnicode_2BYTE_KIND = 2, |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
456 PyUnicode_4BYTE_KIND = 4 |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
457 }; |
| 72475 | 458 |
| 72688 | 459 /* Return pointers to the canonical representation cast to unsigned char, |
| 72475 | 460 Py_UCS2, or Py_UCS4 for direct character access. |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
461 No checks are performed, use PyUnicode_KIND() before to ensure |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
462 these will work correctly. */ |
| 72475 | 463 |
| 464 #define PyUnicode_1BYTE_DATA(op) ((Py_UCS1*)PyUnicode_DATA(op)) | |
| 465 #define PyUnicode_2BYTE_DATA(op) ((Py_UCS2*)PyUnicode_DATA(op)) | |
| 466 #define PyUnicode_4BYTE_DATA(op) ((Py_UCS4*)PyUnicode_DATA(op)) | |
| 467 | |
|
72494
93cc591cffe4
Strip trailing spaces in unicodeobject.[ch]
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72492
diff
changeset
|
468 /* Return one of the PyUnicode_*_KIND values defined above. */ |
| 72475 | 469 #define PyUnicode_KIND(op) \ |
| 470 (assert(PyUnicode_Check(op)), \ | |
| 471 assert(PyUnicode_IS_READY(op)), \ | |
| 472 ((PyASCIIObject *)(op))->state.kind) | |
| 473 | |
|
72494
93cc591cffe4
Strip trailing spaces in unicodeobject.[ch]
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72492
diff
changeset
|
474 /* Return a void pointer to the raw unicode buffer. */ |
| 72475 | 475 #define _PyUnicode_COMPACT_DATA(op) \ |
|
72980
cade7c20c728
Simplify _PyUnicode_COMPACT_DATA() macro
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72976
diff
changeset
|
476 (PyUnicode_IS_ASCII(op) ? \ |
| 72475 | 477 ((void*)((PyASCIIObject*)(op) + 1)) : \ |
| 478 ((void*)((PyCompactUnicodeObject*)(op) + 1))) | |
| 479 | |
| 480 #define _PyUnicode_NONCOMPACT_DATA(op) \ | |
| 481 (assert(((PyUnicodeObject*)(op))->data.any), \ | |
| 482 ((((PyUnicodeObject *)(op))->data.any))) | |
| 483 | |
| 484 #define PyUnicode_DATA(op) \ | |
| 485 (assert(PyUnicode_Check(op)), \ | |
| 486 PyUnicode_IS_COMPACT(op) ? _PyUnicode_COMPACT_DATA(op) : \ | |
| 487 _PyUnicode_NONCOMPACT_DATA(op)) | |
| 488 | |
| 489 /* In the access macros below, "kind" may be evaluated more than once. | |
| 490 All other macro parameters are evaluated exactly once, so it is safe | |
| 491 to put side effects into them (such as increasing the index). */ | |
| 492 | |
| 493 /* Write into the canonical representation, this macro does not do any sanity | |
| 494 checks and is intended for usage in loops. The caller should cache the | |
| 72691 | 495 kind and data pointers obtained from other macro calls. |
| 72475 | 496 index is the index in the string (starts at 0) and value is the new |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
497 code point value which should be written to that location. */ |
| 72475 | 498 #define PyUnicode_WRITE(kind, data, index, value) \ |
| 499 do { \ | |
| 500 switch ((kind)) { \ | |
| 501 case PyUnicode_1BYTE_KIND: { \ | |
| 502 ((Py_UCS1 *)(data))[(index)] = (Py_UCS1)(value); \ | |
| 503 break; \ | |
| 504 } \ | |
| 505 case PyUnicode_2BYTE_KIND: { \ | |
| 506 ((Py_UCS2 *)(data))[(index)] = (Py_UCS2)(value); \ | |
| 507 break; \ | |
| 508 } \ | |
| 509 default: { \ | |
| 510 assert((kind) == PyUnicode_4BYTE_KIND); \ | |
| 511 ((Py_UCS4 *)(data))[(index)] = (Py_UCS4)(value); \ | |
| 512 } \ | |
| 513 } \ | |
| 514 } while (0) | |
| 515 | |
| 72691 | 516 /* Read a code point from the string's canonical representation. No checks |
| 72475 | 517 or ready calls are performed. */ |
| 518 #define PyUnicode_READ(kind, data, index) \ | |
| 519 ((Py_UCS4) \ | |
| 520 ((kind) == PyUnicode_1BYTE_KIND ? \ | |
|
72577
5231de1080b0
Use Py_UCS1 instead of unsigned char in unicodeobject.h
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72575
diff
changeset
|
521 ((const Py_UCS1 *)(data))[(index)] : \ |
| 72475 | 522 ((kind) == PyUnicode_2BYTE_KIND ? \ |
| 523 ((const Py_UCS2 *)(data))[(index)] : \ | |
| 524 ((const Py_UCS4 *)(data))[(index)] \ | |
| 525 ) \ | |
| 526 )) | |
| 527 | |
| 528 /* PyUnicode_READ_CHAR() is less efficient than PyUnicode_READ() because it | |
| 529 calls PyUnicode_KIND() and might call it twice. For single reads, use | |
| 530 PyUnicode_READ_CHAR, for multiple consecutive reads callers should | |
| 531 cache kind and use PyUnicode_READ instead. */ | |
| 532 #define PyUnicode_READ_CHAR(unicode, index) \ | |
|
72598
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
533 (assert(PyUnicode_Check(unicode)), \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
534 assert(PyUnicode_IS_READY(unicode)), \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
535 (Py_UCS4) \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
536 (PyUnicode_KIND((unicode)) == PyUnicode_1BYTE_KIND ? \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
537 ((const Py_UCS1 *)(PyUnicode_DATA((unicode))))[(index)] : \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
538 (PyUnicode_KIND((unicode)) == PyUnicode_2BYTE_KIND ? \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
539 ((const Py_UCS2 *)(PyUnicode_DATA((unicode))))[(index)] : \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
540 ((const Py_UCS4 *)(PyUnicode_DATA((unicode))))[(index)] \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
541 ) \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
542 )) |
| 72475 | 543 |
| 544 /* Returns the length of the unicode string. The caller has to make sure that | |
| 545 the string has it's canonical representation set before calling | |
| 546 this macro. Call PyUnicode_(FAST_)Ready to ensure that. */ | |
| 547 #define PyUnicode_GET_LENGTH(op) \ | |
| 548 (assert(PyUnicode_Check(op)), \ | |
| 549 assert(PyUnicode_IS_READY(op)), \ | |
| 550 ((PyASCIIObject *)(op))->length) | |
| 551 | |
| 552 | |
| 553 /* Fast check to determine whether an object is ready. Equivalent to | |
| 554 PyUnicode_IS_COMPACT(op) || ((PyUnicodeObject*)(op))->data.any) */ | |
| 555 | |
| 556 #define PyUnicode_IS_READY(op) (((PyASCIIObject*)op)->state.ready) | |
| 557 | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
558 /* PyUnicode_READY() does less work than _PyUnicode_Ready() in the best |
| 72475 | 559 case. If the canonical representation is not yet set, it will still call |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
560 _PyUnicode_Ready(). |
| 72475 | 561 Returns 0 on success and -1 on errors. */ |
| 562 #define PyUnicode_READY(op) \ | |
| 563 (assert(PyUnicode_Check(op)), \ | |
| 564 (PyUnicode_IS_READY(op) ? \ | |
|
72529
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
565 0 : _PyUnicode_Ready((PyObject *)(op)))) |
| 72475 | 566 |
| 567 /* Return a maximum character value which is suitable for creating another | |
| 568 string based on op. This is always an approximation but more efficient | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
569 than iterating over the string. */ |
| 72475 | 570 #define PyUnicode_MAX_CHAR_VALUE(op) \ |
| 571 (assert(PyUnicode_IS_READY(op)), \ | |
|
72904
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
572 (PyUnicode_IS_ASCII(op) ? \ |
|
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
573 (0x7f) : \ |
| 72475 | 574 (PyUnicode_KIND(op) == PyUnicode_1BYTE_KIND ? \ |
|
72904
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
575 (0xffU) : \ |
| 72475 | 576 (PyUnicode_KIND(op) == PyUnicode_2BYTE_KIND ? \ |
|
72904
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
577 (0xffffU) : \ |
|
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
578 (0x10ffffU))))) |
| 72475 | 579 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
580 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
581 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
582 /* --- Constants ---------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
583 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
584 /* This Unicode character will be used as replacement character during |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
585 decoding if the errors argument is set to "replace". Note: the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
586 Unicode character U+FFFD is the official REPLACEMENT CHARACTER in |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
587 Unicode 3.0. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
588 |
|
72488
17626ba912fb
Set Py_UNICODE_REPLACEMENT_CHARACTER type to Py_UCS4, instead of Py_UNICODE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72475
diff
changeset
|
589 #define Py_UNICODE_REPLACEMENT_CHARACTER ((Py_UCS4) 0xFFFD) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
590 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
591 /* === Public API ========================================================= */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
592 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
593 /* --- Plain Py_UNICODE --------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
594 |
| 72475 | 595 /* With PEP 393, this is the recommended way to allocate a new unicode object. |
| 596 This function will allocate the object and its buffer in a single memory | |
| 597 block. Objects created using this function are not resizable. */ | |
| 598 #ifndef Py_LIMITED_API | |
| 599 PyAPI_FUNC(PyObject*) PyUnicode_New( | |
| 600 Py_ssize_t size, /* Number of code points in the new string */ | |
| 601 Py_UCS4 maxchar /* maximum code point value in the string */ | |
| 602 ); | |
| 603 #endif | |
| 604 | |
|
72529
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
605 /* Initializes the canonical string representation from a the deprecated |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
606 wstr/Py_UNICODE representation. This function is used to convert Unicode |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
607 objects which were created using the old API to the new flexible format |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
608 introduced with PEP 393. |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
609 |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
610 Don't call this function directly, use the public PyUnicode_READY() macro |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
611 instead. */ |
| 72475 | 612 #ifndef Py_LIMITED_API |
| 613 PyAPI_FUNC(int) _PyUnicode_Ready( | |
|
72529
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
614 PyObject *unicode /* Unicode object */ |
| 72475 | 615 ); |
| 616 #endif | |
| 617 | |
|
72546
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
618 /* Get a copy of a Unicode string. */ |
|
73942
9c76bb654809
Make PyUnicode_Copy() private => _PyUnicode_Copy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73941
diff
changeset
|
619 #ifndef Py_LIMITED_API |
|
9c76bb654809
Make PyUnicode_Copy() private => _PyUnicode_Copy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73941
diff
changeset
|
620 PyAPI_FUNC(PyObject*) _PyUnicode_Copy( |
|
72546
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
621 PyObject *unicode |
|
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
622 ); |
|
73942
9c76bb654809
Make PyUnicode_Copy() private => _PyUnicode_Copy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73941
diff
changeset
|
623 #endif |
|
72546
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
624 |
| 72475 | 625 /* Copy character from one unicode object into another, this function performs |
|
74261
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
626 character conversion when necessary and falls back to memcpy() if possible. |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
627 |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
628 Fail if to is too small (smaller than *how_many* or smaller than |
|
72527
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
629 len(from)-from_start), or if kind(from[from_start:from_start+how_many]) > |
|
74261
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
630 kind(to), or if *to* has more than 1 reference. |
|
72492
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
631 |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
632 Return the number of written character, or return -1 and raise an exception |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
633 on error. |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
634 |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
635 Pseudo-code: |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
636 |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
637 how_many = min(how_many, len(from) - from_start) |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
638 to[to_start:to_start+how_many] = from[from_start:from_start+how_many] |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
639 return how_many |
|
72527
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
640 |
|
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
641 Note: The function doesn't write a terminating null character. |
|
72492
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
642 */ |
| 72475 | 643 #ifndef Py_LIMITED_API |
|
72492
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
644 PyAPI_FUNC(Py_ssize_t) PyUnicode_CopyCharacters( |
| 72475 | 645 PyObject *to, |
| 646 Py_ssize_t to_start, | |
| 647 PyObject *from, | |
| 648 Py_ssize_t from_start, | |
| 649 Py_ssize_t how_many | |
| 650 ); | |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
651 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
652 /* Unsafe version of PyUnicode_CopyCharacters(): don't check arguments and so |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
653 may crash if parameters are invalid (e.g. if the output string |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
654 is too short). */ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
655 PyAPI_FUNC(void) _PyUnicode_FastCopyCharacters( |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
656 PyObject *to, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
657 Py_ssize_t to_start, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
658 PyObject *from, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
659 Py_ssize_t from_start, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
660 Py_ssize_t how_many |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
661 ); |
| 72475 | 662 #endif |
| 663 | |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
664 #ifndef Py_LIMITED_API |
|
74261
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
665 /* Fill a string with a character: write fill_char into |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
666 unicode[start:start+length]. |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
667 |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
668 Fail if fill_char is bigger than the string maximum character, or if the |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
669 string has more than 1 reference. |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
670 |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
671 Return the number of written character, or return -1 and raise an exception |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
672 on error. */ |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
673 PyAPI_FUNC(Py_ssize_t) PyUnicode_Fill( |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
674 PyObject *unicode, |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
675 Py_ssize_t start, |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
676 Py_ssize_t length, |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
677 Py_UCS4 fill_char |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
678 ); |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
679 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
680 /* Unsafe version of PyUnicode_Fill(): don't check arguments and so may crash |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
681 if parameters are invalid (e.g. if length is longer than the string). */ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
682 PyAPI_FUNC(void) _PyUnicode_FastFill( |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
683 PyObject *unicode, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
684 Py_ssize_t start, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
685 Py_ssize_t length, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
686 Py_UCS4 fill_char |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
687 ); |
|
74261
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
688 #endif |
|
405f65676010
Add a new PyUnicode_Fill() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74124
diff
changeset
|
689 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
690 /* Create a Unicode Object from the Py_UNICODE buffer u of the given |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
691 size. |
|
17618
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
692 |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
693 u may be NULL which causes the contents to be undefined. It is the |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
694 user's responsibility to fill in the needed data afterwards. Note |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
695 that modifying the Unicode object contents after construction is |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
696 only allowed if u was set to NULL. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
697 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
698 The buffer is copied into the new object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
699 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
700 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
701 PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
702 const Py_UNICODE *u, /* Unicode buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
703 Py_ssize_t size /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
704 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
705 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
706 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
707 /* Similar to PyUnicode_FromUnicode(), but u points to UTF-8 encoded bytes */ |
|
41766
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
708 PyAPI_FUNC(PyObject*) PyUnicode_FromStringAndSize( |
|
67152
f61196453c78
Issue #9738: Ooops, fix typos in my previous commit (r87506)
Victor Stinner <victor.stinner@haypocalc.com>
parents:
67150
diff
changeset
|
709 const char *u, /* UTF-8 encoded string */ |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
710 Py_ssize_t size /* size of buffer */ |
|
41766
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
711 ); |
|
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
712 |
|
41619
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
713 /* Similar to PyUnicode_FromUnicode(), but u points to null-terminated |
| 72475 | 714 UTF-8 encoded bytes. The size is determined with strlen(). */ |
|
41619
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
715 PyAPI_FUNC(PyObject*) PyUnicode_FromString( |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
716 const char *u /* UTF-8 encoded string */ |
|
41619
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
717 ); |
|
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
718 |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
719 #ifndef Py_LIMITED_API |
|
72685
61ca5262b6c4
Speedup str[a:b] and PyUnicode_FromKindAndData
Victor Stinner <vstinner@wyplay.com>
parents:
72632
diff
changeset
|
720 /* Create a new string from a buffer of Py_UCS1, Py_UCS2 or Py_UCS4 characters. |
|
61ca5262b6c4
Speedup str[a:b] and PyUnicode_FromKindAndData
Victor Stinner <vstinner@wyplay.com>
parents:
72632
diff
changeset
|
721 Scan the string to find the maximum character. */ |
| 72475 | 722 PyAPI_FUNC(PyObject*) PyUnicode_FromKindAndData( |
| 723 int kind, | |
| 724 const void *buffer, | |
| 725 Py_ssize_t size); | |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
726 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
727 /* Create a new string from a buffer of ASCII characters. |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
728 WARNING: Don't check if the string contains any non-ASCII character. */ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
729 PyAPI_FUNC(PyObject*) _PyUnicode_FromASCII( |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
730 const char *buffer, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
731 Py_ssize_t size); |
| 72475 | 732 #endif |
| 733 | |
| 734 PyAPI_FUNC(PyObject*) PyUnicode_Substring( | |
| 735 PyObject *str, | |
| 736 Py_ssize_t start, | |
| 737 Py_ssize_t end); | |
| 738 | |
|
76509
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
739 #ifndef Py_LIMITED_API |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
740 /* Compute the maximum character of the substring unicode[start:end]. |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
741 Return 127 for an empty string. */ |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
742 PyAPI_FUNC(Py_UCS4) _PyUnicode_FindMaxChar ( |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
743 PyObject *unicode, |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
744 Py_ssize_t start, |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
745 Py_ssize_t end); |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
746 #endif |
|
139c3ae84772
Close #14648: Compute correctly maxchar in str.format() for substrin
Victor Stinner <victor.stinner@gmail.com>
parents:
75378
diff
changeset
|
747 |
|
72785
fe65c75c9f9d
Update C API docs for PEP 393.
Georg Brandl <georg@python.org>
parents:
72752
diff
changeset
|
748 /* Copy the string into a UCS4 buffer including the null character if copy_null |
| 72475 | 749 is set. Return NULL and raise an exception on error. Raise a ValueError if |
| 750 the buffer is smaller than the string. Return buffer on success. | |
| 751 | |
| 752 buflen is the length of the buffer in (Py_UCS4) characters. */ | |
| 753 PyAPI_FUNC(Py_UCS4*) PyUnicode_AsUCS4( | |
| 754 PyObject *unicode, | |
| 755 Py_UCS4* buffer, | |
| 756 Py_ssize_t buflen, | |
| 757 int copy_null); | |
| 758 | |
| 759 /* Copy the string into a UCS4 buffer. A new buffer is allocated using | |
| 760 * PyMem_Malloc; if this fails, NULL is returned with a memory error | |
| 761 exception set. */ | |
| 762 PyAPI_FUNC(Py_UCS4*) PyUnicode_AsUCS4Copy(PyObject *unicode); | |
| 763 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
764 /* Return a read-only pointer to the Unicode object's internal |
| 72475 | 765 Py_UNICODE buffer. |
| 766 If the wchar_t/Py_UNICODE representation is not yet available, this | |
| 767 function will calculate it. */ | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
768 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
769 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
770 PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
771 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
772 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
773 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
774 |
| 72475 | 775 /* Return a read-only pointer to the Unicode object's internal |
| 776 Py_UNICODE buffer and save the length at size. | |
| 777 If the wchar_t/Py_UNICODE representation is not yet available, this | |
| 778 function will calculate it. */ | |
| 779 | |
| 780 #ifndef Py_LIMITED_API | |
| 781 PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicodeAndSize( | |
| 782 PyObject *unicode, /* Unicode object */ | |
| 783 Py_ssize_t *size /* location where to save the length */ | |
| 784 ); | |
| 785 #endif | |
| 786 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
787 /* Get the length of the Unicode object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
788 |
| 72475 | 789 PyAPI_FUNC(Py_ssize_t) PyUnicode_GetLength( |
| 790 PyObject *unicode | |
| 791 ); | |
| 792 | |
|
72494
93cc591cffe4
Strip trailing spaces in unicodeobject.[ch]
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72492
diff
changeset
|
793 /* Get the number of Py_UNICODE units in the |
| 72475 | 794 string representation. */ |
| 795 | |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
796 PyAPI_FUNC(Py_ssize_t) PyUnicode_GetSize( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
797 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
798 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
799 |
| 72475 | 800 /* Read a character from the string. */ |
| 801 | |
| 802 PyAPI_FUNC(Py_UCS4) PyUnicode_ReadChar( | |
| 803 PyObject *unicode, | |
| 804 Py_ssize_t index | |
| 805 ); | |
| 806 | |
| 807 /* Write a character to the string. The string must have been created through | |
|
72575
99aa46107a22
PyUnicode_WriteChar() raises IndexError on invalid index
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72559
diff
changeset
|
808 PyUnicode_New, must not be shared, and must not have been hashed yet. |
|
99aa46107a22
PyUnicode_WriteChar() raises IndexError on invalid index
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72559
diff
changeset
|
809 |
|
99aa46107a22
PyUnicode_WriteChar() raises IndexError on invalid index
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72559
diff
changeset
|
810 Return 0 on success, -1 on error. */ |
| 72475 | 811 |
| 812 PyAPI_FUNC(int) PyUnicode_WriteChar( | |
| 813 PyObject *unicode, | |
| 814 Py_ssize_t index, | |
| 815 Py_UCS4 character | |
| 816 ); | |
| 817 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
818 #ifndef Py_LIMITED_API |
|
18298
982163c4e9d2
Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents:
18296
diff
changeset
|
819 /* Get the maximum ordinal for a Unicode character. */ |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
820 PyAPI_FUNC(Py_UNICODE) PyUnicode_GetMax(void); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
821 #endif |
|
18298
982163c4e9d2
Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents:
18296
diff
changeset
|
822 |
|
73946
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
823 /* Resize an Unicode object. The length is the number of characters, except |
|
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
824 if the kind of the string is PyUnicode_WCHAR_KIND: in this case, the length |
|
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
825 is the number of Py_UNICODE characters. |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
826 |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
827 *unicode is modified to point to the new (resized) object and 0 |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
828 returned on success. |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
829 |
|
73946
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
830 Try to resize the string in place (which is usually faster than allocating |
|
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
831 a new string and copy characters), or create a new string. |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
832 |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
833 Error handling is implemented as follows: an exception is set, -1 |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
834 is returned and *unicode left untouched. |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
835 |
|
73947
3ad7d01acbf4
PyUnicode_Resize(): warn about canonical representation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73946
diff
changeset
|
836 WARNING: The function doesn't check string content, the result may not be a |
|
3ad7d01acbf4
PyUnicode_Resize(): warn about canonical representation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73946
diff
changeset
|
837 string in canonical representation. */ |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
838 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
839 PyAPI_FUNC(int) PyUnicode_Resize( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
840 PyObject **unicode, /* Pointer to the Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
841 Py_ssize_t length /* New length */ |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
842 ); |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
843 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
844 /* Coerce obj to an Unicode object and return a reference with |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
845 *incremented* refcount. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
846 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
847 Coercion is done in the following way: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
848 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
849 1. bytes, bytearray and other char buffer compatible objects are decoded |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
850 under the assumptions that they contain data using the UTF-8 |
|
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
851 encoding. Decoding is done in "strict" mode. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
852 |
|
20554
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
853 2. All other objects (including Unicode objects) raise an |
|
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
854 exception. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
855 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
856 The API returns NULL in case of an error. The caller is responsible |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
857 for decref'ing the returned objects. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
858 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
859 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
860 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
861 PyAPI_FUNC(PyObject*) PyUnicode_FromEncodedObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
862 register PyObject *obj, /* Object */ |
|
13401
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
863 const char *encoding, /* encoding */ |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
864 const char *errors /* error handling */ |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
865 ); |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
866 |
|
20554
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
867 /* Coerce obj to an Unicode object and return a reference with |
|
13401
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
868 *incremented* refcount. |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
869 |
|
20554
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
870 Unicode objects are passed back as-is (subclasses are converted to |
|
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
871 true Unicode objects), all other objects are delegated to |
|
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
872 PyUnicode_FromEncodedObject(obj, NULL, "strict") which results in |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
873 using UTF-8 encoding as basis for decoding the object. |
|
13401
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
874 |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
875 The API returns NULL in case of an error. The caller is responsible |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
876 for decref'ing the returned objects. |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
877 |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
878 */ |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
879 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
880 PyAPI_FUNC(PyObject*) PyUnicode_FromObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
881 register PyObject *obj /* Object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
882 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
883 |
|
64639
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
884 PyAPI_FUNC(PyObject *) PyUnicode_FromFormatV( |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
885 const char *format, /* ASCII-encoded string */ |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
886 va_list vargs |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
887 ); |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
888 PyAPI_FUNC(PyObject *) PyUnicode_FromFormat( |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
889 const char *format, /* ASCII-encoded string */ |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
890 ... |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
891 ); |
|
41766
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
892 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
893 #ifndef Py_LIMITED_API |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
894 typedef struct { |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
895 PyObject *buffer; |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
896 void *data; |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
897 enum PyUnicode_Kind kind; |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
898 Py_UCS4 maxchar; |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
899 Py_ssize_t size; |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
900 Py_ssize_t pos; |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
901 /* minimum length of the buffer when overallocation is enabled, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
902 see _PyUnicodeWriter_Init() */ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
903 Py_ssize_t min_length; |
|
77351
09736ae1c314
Issue #14993: Use standard "unsigned char" instead of a unsigned char bitfield
Victor Stinner <victor.stinner@gmail.com>
parents:
77223
diff
changeset
|
904 unsigned char overallocate; |
|
09736ae1c314
Issue #14993: Use standard "unsigned char" instead of a unsigned char bitfield
Victor Stinner <victor.stinner@gmail.com>
parents:
77223
diff
changeset
|
905 /* If readonly is 1, buffer is a shared string (cannot be modified) |
|
09736ae1c314
Issue #14993: Use standard "unsigned char" instead of a unsigned char bitfield
Victor Stinner <victor.stinner@gmail.com>
parents:
77223
diff
changeset
|
906 and size is set to 0. */ |
|
09736ae1c314
Issue #14993: Use standard "unsigned char" instead of a unsigned char bitfield
Victor Stinner <victor.stinner@gmail.com>
parents:
77223
diff
changeset
|
907 unsigned char readonly; |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
908 } _PyUnicodeWriter ; |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
909 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
910 /* Initialize a Unicode writer. |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
911 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
912 If min_length is greater than zero, _PyUnicodeWriter_Prepare() |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
913 overallocates the buffer and min_length is the minimum length in characters |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
914 of the buffer. */ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
915 PyAPI_FUNC(void) |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
916 _PyUnicodeWriter_Init(_PyUnicodeWriter *writer, Py_ssize_t min_length); |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
917 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
918 /* Prepare the buffer to write 'length' characters |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
919 with the specified maximum character. |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
920 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
921 Return 0 on success, raise an exception and return -1 on error. */ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
922 #define _PyUnicodeWriter_Prepare(WRITER, LENGTH, MAXCHAR) \ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
923 (((MAXCHAR) <= (WRITER)->maxchar \ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
924 && (LENGTH) <= (WRITER)->size - (WRITER)->pos) \ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
925 ? 0 \ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
926 : (((LENGTH) == 0) \ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
927 ? 0 \ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
928 : _PyUnicodeWriter_PrepareInternal((WRITER), (LENGTH), (MAXCHAR)))) |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
929 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
930 /* Don't call this function directly, use the _PyUnicodeWriter_Prepare() macro |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
931 instead. */ |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
932 PyAPI_FUNC(int) |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
933 _PyUnicodeWriter_PrepareInternal(_PyUnicodeWriter *writer, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
934 Py_ssize_t length, Py_UCS4 maxchar); |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
935 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
936 PyAPI_FUNC(int) |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
937 _PyUnicodeWriter_WriteStr(_PyUnicodeWriter *writer, PyObject *str); |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
938 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
939 PyAPI_FUNC(PyObject *) |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
940 _PyUnicodeWriter_Finish(_PyUnicodeWriter *writer); |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
941 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
942 PyAPI_FUNC(void) |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
943 _PyUnicodeWriter_Dealloc(_PyUnicodeWriter *writer); |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
944 #endif |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
945 |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
946 #ifndef Py_LIMITED_API |
|
47393
0da27fdf0192
Refactor and clean up str.format() code (and helpers) in advance of optimizations.
Eric Smith <eric@trueblade.com>
parents:
46909
diff
changeset
|
947 /* Format the object based on the format_spec, as defined in PEP 3101 |
|
0da27fdf0192
Refactor and clean up str.format() code (and helpers) in advance of optimizations.
Eric Smith <eric@trueblade.com>
parents:
46909
diff
changeset
|
948 (Advanced String Formatting). */ |
|
77223
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
949 PyAPI_FUNC(int) _PyUnicode_FormatAdvancedWriter( |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
950 _PyUnicodeWriter *writer, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
951 PyObject *obj, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
952 PyObject *format_spec, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
953 Py_ssize_t start, |
|
22b56b0b8619
Issue #14744: Use the new _PyUnicodeWriter internal API to speed up str%args and str.format(args)
Victor Stinner <victor.stinner@gmail.com>
parents:
76509
diff
changeset
|
954 Py_ssize_t end); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
955 #endif |
|
47393
0da27fdf0192
Refactor and clean up str.format() code (and helpers) in advance of optimizations.
Eric Smith <eric@trueblade.com>
parents:
46909
diff
changeset
|
956 |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
957 PyAPI_FUNC(void) PyUnicode_InternInPlace(PyObject **); |
|
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
958 PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **); |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
959 PyAPI_FUNC(PyObject *) PyUnicode_InternFromString( |
|
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
960 const char *u /* UTF-8 encoded string */ |
|
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
961 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
962 #ifndef Py_LIMITED_API |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
963 PyAPI_FUNC(void) _Py_ReleaseInternedUnicodeStrings(void); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
964 #endif |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
965 |
|
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
966 /* Use only if you know it's a string */ |
| 72475 | 967 #define PyUnicode_CHECK_INTERNED(op) \ |
| 968 (((PyASCIIObject *)(op))->state.interned) | |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
969 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
970 /* --- wchar_t support for platforms which support it --------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
971 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
972 #ifdef HAVE_WCHAR_H |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
973 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
974 /* Create a Unicode Object from the wchar_t buffer w of the given |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
975 size. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
976 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
977 The buffer is copied into the new object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
978 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
979 PyAPI_FUNC(PyObject*) PyUnicode_FromWideChar( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
980 register const wchar_t *w, /* wchar_t buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
981 Py_ssize_t size /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
982 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
983 |
|
34016
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
984 /* Copies the Unicode Object contents into the wchar_t buffer w. At |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
985 most size wchar_t characters are copied. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
986 |
|
34016
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
987 Note that the resulting wchar_t string may or may not be |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
988 0-terminated. It is the responsibility of the caller to make sure |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
989 that the wchar_t string is 0-terminated in case this is required by |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
990 the application. |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
991 |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
992 Returns the number of wchar_t characters copied (excluding a |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
993 possibly trailing 0-termination character) or -1 in case of an |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
994 error. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
995 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
996 PyAPI_FUNC(Py_ssize_t) PyUnicode_AsWideChar( |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
997 PyObject *unicode, /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
998 register wchar_t *w, /* wchar_t buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
999 Py_ssize_t size /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1000 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1001 |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1002 /* Convert the Unicode object to a wide character string. The output string |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1003 always ends with a nul character. If size is not NULL, write the number of |
|
72298
028423c89f8d
Fix PyUnicode_AsWideCharString() doc: size doesn't contain the null character
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68257
diff
changeset
|
1004 wide characters (excluding the null character) into *size. |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1005 |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1006 Returns a buffer allocated by PyMem_Alloc() (use PyMem_Free() to free it) |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1007 on success. On error, returns NULL, *size is undefined and raises a |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1008 MemoryError. */ |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1009 |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1010 PyAPI_FUNC(wchar_t*) PyUnicode_AsWideCharString( |
|
65157
e3ae5b00ba15
PyUnicode_AsWideCharString() takes a PyObject*, not a PyUnicodeObject*
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65007
diff
changeset
|
1011 PyObject *unicode, /* Unicode object */ |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1012 Py_ssize_t *size /* number of characters of the result */ |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1013 ); |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
1014 |
|
72559
fc7d2c6db61b
_PyUnicode_AsKind() is *not* part of the stable ABI
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72554
diff
changeset
|
1015 #ifndef Py_LIMITED_API |
| 72475 | 1016 PyAPI_FUNC(void*) _PyUnicode_AsKind(PyObject *s, unsigned int kind); |
|
72559
fc7d2c6db61b
_PyUnicode_AsKind() is *not* part of the stable ABI
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72554
diff
changeset
|
1017 #endif |
| 72475 | 1018 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1019 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1020 |
|
24737
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
1021 /* --- Unicode ordinals --------------------------------------------------- */ |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
1022 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1023 /* Create a Unicode Object from the given Unicode code point ordinal. |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1024 |
|
79477
a1aa13ef00c5
#16127: remove outdated references to narrow builds. Patch by Serhiy Storchaka.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
77448
diff
changeset
|
1025 The ordinal must be in range(0x110000). A ValueError is |
|
24737
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
1026 raised in case it is not. |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
1027 |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
1028 */ |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
1029 |
|
24753
27ae978bee58
Add name mangling for new PyUnicode_FromOrdinal() and fix declaration
Marc-André Lemburg <mal@egenix.com>
parents:
24752
diff
changeset
|
1030 PyAPI_FUNC(PyObject*) PyUnicode_FromOrdinal(int ordinal); |
|
24737
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
1031 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1032 /* --- Free-list management ----------------------------------------------- */ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1033 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1034 /* Clear the free list used by the Unicode implementation. |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1035 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1036 This can be used to release memory used for objects on the free |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1037 list back to the Python memory allocator. |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1038 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1039 */ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1040 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1041 PyAPI_FUNC(int) PyUnicode_ClearFreeList(void); |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1042 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1043 /* === Builtin Codecs ===================================================== |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1044 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1045 Many of these APIs take two arguments encoding and errors. These |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1046 parameters encoding and errors have the same semantics as the ones |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
1047 of the builtin str() API. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1048 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
1049 Setting encoding to NULL causes the default encoding (UTF-8) to be used. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1050 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1051 Error handling is set by errors which may also be set to NULL |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1052 meaning to use the default handling defined for the codec. Default |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1053 error handling for all builtin codecs is "strict" (ValueErrors are |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1054 raised). |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1055 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1056 The codecs all use a similar interface. Only deviation from the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1057 generic ones are documented. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1058 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1059 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1060 |
|
12643
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
1061 /* --- Manage the default encoding ---------------------------------------- */ |
|
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
1062 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
1063 /* Returns a pointer to the default encoding (UTF-8) of the |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1064 Unicode object unicode and the size of the encoded representation |
|
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1065 in bytes stored in *size. |
|
43761
658cd34a8c5d
Fixed a bug in PyUnicode_DecodeFSDefault. strcmp() returns 0 on success.
Christian Heimes <christian@cheimes.de>
parents:
43569
diff
changeset
|
1066 |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1067 In case of an error, no *size is set. |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
1068 |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
1069 This function caches the UTF-8 encoded string in the unicodeobject |
|
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
1070 and subsequent calls will return the same string. The memory is released |
| 72475 | 1071 when the unicodeobject is deallocated. |
| 1072 | |
| 1073 _PyUnicode_AsStringAndSize is a #define for PyUnicode_AsUTF8AndSize to | |
| 1074 support the previous internal function with the same behaviour. | |
| 1075 | |
|
48660
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1076 *** This API is for interpreter INTERNAL USE ONLY and will likely |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
1077 *** be removed or changed in the future. |
|
48660
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1078 |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1079 *** If you need to access the Unicode object as UTF-8 bytes string, |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1080 *** please use PyUnicode_AsUTF8String() instead. |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1081 */ |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1082 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1083 #ifndef Py_LIMITED_API |
| 72475 | 1084 PyAPI_FUNC(char *) PyUnicode_AsUTF8AndSize( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1085 PyObject *unicode, |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1086 Py_ssize_t *size); |
| 72475 | 1087 #define _PyUnicode_AsStringAndSize PyUnicode_AsUTF8AndSize |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1088 #endif |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
1089 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
1090 /* Returns a pointer to the default encoding (UTF-8) of the |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1091 Unicode object unicode. |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
1092 |
| 72475 | 1093 Like PyUnicode_AsUTF8AndSize(), this also caches the UTF-8 representation |
| 1094 in the unicodeobject. | |
| 1095 | |
| 1096 _PyUnicode_AsString is a #define for PyUnicode_AsUTF8 to | |
| 1097 support the previous internal function with the same behaviour. | |
| 1098 | |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1099 Use of this API is DEPRECATED since no size information can be |
|
48660
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1100 extracted from the returned data. |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1101 |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1102 *** This API is for interpreter INTERNAL USE ONLY and will likely |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1103 *** be removed or changed for Python 3.1. |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1104 |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1105 *** If you need to access the Unicode object as UTF-8 bytes string, |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1106 *** please use PyUnicode_AsUTF8String() instead. |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
1107 |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1108 */ |
|
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1109 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1110 #ifndef Py_LIMITED_API |
| 72475 | 1111 PyAPI_FUNC(char *) PyUnicode_AsUTF8(PyObject *unicode); |
| 1112 #define _PyUnicode_AsString PyUnicode_AsUTF8 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1113 #endif |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1114 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
1115 /* Returns "utf-8". */ |
|
12643
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
1116 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1117 PyAPI_FUNC(const char*) PyUnicode_GetDefaultEncoding(void); |
|
12643
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
1118 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1119 /* --- Generic Codecs ----------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1120 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1121 /* Create a Unicode object by decoding the encoded string s of the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1122 given size. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1123 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1124 PyAPI_FUNC(PyObject*) PyUnicode_Decode( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1125 const char *s, /* encoded string */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1126 Py_ssize_t size, /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1127 const char *encoding, /* encoding */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1128 const char *errors /* error handling */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1129 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1130 |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1131 /* Decode a Unicode object unicode and return the result as Python |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1132 object. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1133 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1134 PyAPI_FUNC(PyObject*) PyUnicode_AsDecodedObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1135 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1136 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1137 const char *errors /* error handling */ |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1138 ); |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1139 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1140 /* Decode a Unicode object unicode and return the result as Unicode |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1141 object. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1142 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1143 PyAPI_FUNC(PyObject*) PyUnicode_AsDecodedUnicode( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1144 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1145 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1146 const char *errors /* error handling */ |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1147 ); |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1148 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1149 /* Encodes a Py_UNICODE buffer of the given size and returns a |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1150 Python string object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1151 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1152 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1153 PyAPI_FUNC(PyObject*) PyUnicode_Encode( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1154 const Py_UNICODE *s, /* Unicode char buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1155 Py_ssize_t size, /* number of Py_UNICODE chars to encode */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1156 const char *encoding, /* encoding */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1157 const char *errors /* error handling */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1158 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1159 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1160 |
|
32551
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1161 /* Encodes a Unicode object and returns the result as Python |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1162 object. */ |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1163 |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1164 PyAPI_FUNC(PyObject*) PyUnicode_AsEncodedObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1165 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1166 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1167 const char *errors /* error handling */ |
|
32551
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1168 ); |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1169 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1170 /* Encodes a Unicode object and returns the result as Python string |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1171 object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1172 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1173 PyAPI_FUNC(PyObject*) PyUnicode_AsEncodedString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1174 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1175 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1176 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1177 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1178 |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1179 /* Encodes a Unicode object and returns the result as Unicode |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1180 object. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1181 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1182 PyAPI_FUNC(PyObject*) PyUnicode_AsEncodedUnicode( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1183 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1184 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1185 const char *errors /* error handling */ |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1186 ); |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1187 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1188 /* Build an encoding map. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1189 |
|
38673
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1190 PyAPI_FUNC(PyObject*) PyUnicode_BuildEncodingMap( |
|
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1191 PyObject* string /* 256 character map */ |
|
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1192 ); |
|
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1193 |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1194 /* --- UTF-7 Codecs ------------------------------------------------------- */ |
|
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1195 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1196 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF7( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1197 const char *string, /* UTF-7 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1198 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1199 const char *errors /* error handling */ |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1200 ); |
|
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1201 |
|
43933
9c4bf18e5d30
Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
43761
diff
changeset
|
1202 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF7Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1203 const char *string, /* UTF-7 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1204 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1205 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1206 Py_ssize_t *consumed /* bytes consumed */ |
|
43933
9c4bf18e5d30
Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
43761
diff
changeset
|
1207 ); |
|
9c4bf18e5d30
Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
43761
diff
changeset
|
1208 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1209 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1210 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF7( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1211 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1212 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1213 int base64SetO, /* Encode RFC2152 Set O characters in base64 */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1214 int base64WhiteSpace, /* Encode whitespace (sp, ht, nl, cr) in base64 */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1215 const char *errors /* error handling */ |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1216 ); |
|
73476
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1217 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF7( |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1218 PyObject *unicode, /* Unicode object */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1219 int base64SetO, /* Encode RFC2152 Set O characters in base64 */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1220 int base64WhiteSpace, /* Encode whitespace (sp, ht, nl, cr) in base64 */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1221 const char *errors /* error handling */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1222 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1223 #endif |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1224 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1225 /* --- UTF-8 Codecs ------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1226 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1227 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1228 const char *string, /* UTF-8 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1229 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1230 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1231 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1232 |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1233 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1234 const char *string, /* UTF-8 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1235 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1236 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1237 Py_ssize_t *consumed /* bytes consumed */ |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1238 ); |
|
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1239 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1240 PyAPI_FUNC(PyObject*) PyUnicode_AsUTF8String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1241 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1242 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1243 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1244 #ifndef Py_LIMITED_API |
| 72475 | 1245 PyAPI_FUNC(PyObject*) _PyUnicode_AsUTF8String( |
| 1246 PyObject *unicode, | |
| 1247 const char *errors); | |
| 1248 | |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1249 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF8( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1250 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1251 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1252 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1253 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1254 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1255 |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1256 /* --- UTF-32 Codecs ------------------------------------------------------ */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1257 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1258 /* Decodes length bytes from a UTF-32 encoded buffer string and returns |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1259 the corresponding Unicode object. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1260 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1261 errors (if non-NULL) defines the error handling. It defaults |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1262 to "strict". |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1263 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1264 If byteorder is non-NULL, the decoder starts decoding using the |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1265 given byte order: |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1266 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1267 *byteorder == -1: little endian |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1268 *byteorder == 0: native order |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1269 *byteorder == 1: big endian |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1270 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1271 In native mode, the first four bytes of the stream are checked for a |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1272 BOM mark. If found, the BOM mark is analysed, the byte order |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1273 adjusted and the BOM skipped. In the other modes, no BOM mark |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1274 interpretation is done. After completion, *byteorder is set to the |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1275 current byte order at the end of input data. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1276 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1277 If byteorder is NULL, the codec starts in native order mode. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1278 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1279 */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1280 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1281 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF32( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1282 const char *string, /* UTF-32 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1283 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1284 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1285 int *byteorder /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1286 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1287 exit */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1288 ); |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1289 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1290 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF32Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1291 const char *string, /* UTF-32 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1292 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1293 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1294 int *byteorder, /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1295 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1296 exit */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1297 Py_ssize_t *consumed /* bytes consumed */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1298 ); |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1299 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1300 /* Returns a Python string using the UTF-32 encoding in native byte |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1301 order. The string always starts with a BOM mark. */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1302 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1303 PyAPI_FUNC(PyObject*) PyUnicode_AsUTF32String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1304 PyObject *unicode /* Unicode object */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1305 ); |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1306 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1307 /* Returns a Python string object holding the UTF-32 encoded value of |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1308 the Unicode data. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1309 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1310 If byteorder is not 0, output is written according to the following |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1311 byte order: |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1312 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1313 byteorder == -1: little endian |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1314 byteorder == 0: native byte order (writes a BOM mark) |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1315 byteorder == 1: big endian |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1316 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1317 If byteorder is 0, the output string will always start with the |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1318 Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1319 prepended. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1320 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1321 */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1322 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1323 #ifndef Py_LIMITED_API |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1324 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF32( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1325 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1326 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1327 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1328 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1329 ); |
|
73476
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1330 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF32( |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1331 PyObject *object, /* Unicode object */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1332 const char *errors, /* error handling */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1333 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1334 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1335 #endif |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1336 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1337 /* --- UTF-16 Codecs ------------------------------------------------------ */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1338 |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1339 /* Decodes length bytes from a UTF-16 encoded buffer string and returns |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1340 the corresponding Unicode object. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1341 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1342 errors (if non-NULL) defines the error handling. It defaults |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1343 to "strict". |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1344 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1345 If byteorder is non-NULL, the decoder starts decoding using the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1346 given byte order: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1347 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1348 *byteorder == -1: little endian |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1349 *byteorder == 0: native order |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1350 *byteorder == 1: big endian |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1351 |
|
17887
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1352 In native mode, the first two bytes of the stream are checked for a |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1353 BOM mark. If found, the BOM mark is analysed, the byte order |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1354 adjusted and the BOM skipped. In the other modes, no BOM mark |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1355 interpretation is done. After completion, *byteorder is set to the |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1356 current byte order at the end of input data. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1357 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1358 If byteorder is NULL, the codec starts in native order mode. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1359 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1360 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1361 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1362 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF16( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1363 const char *string, /* UTF-16 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1364 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1365 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1366 int *byteorder /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1367 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1368 exit */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1369 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1370 |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1371 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF16Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1372 const char *string, /* UTF-16 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1373 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1374 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1375 int *byteorder, /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1376 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1377 exit */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1378 Py_ssize_t *consumed /* bytes consumed */ |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1379 ); |
|
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1380 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1381 /* Returns a Python string using the UTF-16 encoding in native byte |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1382 order. The string always starts with a BOM mark. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1383 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1384 PyAPI_FUNC(PyObject*) PyUnicode_AsUTF16String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1385 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1386 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1387 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1388 /* Returns a Python string object holding the UTF-16 encoded value of |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1389 the Unicode data. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1390 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1391 If byteorder is not 0, output is written according to the following |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1392 byte order: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1393 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1394 byteorder == -1: little endian |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1395 byteorder == 0: native byte order (writes a BOM mark) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1396 byteorder == 1: big endian |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1397 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1398 If byteorder is 0, the output string will always start with the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1399 Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1400 prepended. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1401 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1402 Note that Py_UNICODE data is being interpreted as UTF-16 reduced to |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1403 UCS-2. This trick makes it possible to add full UTF-16 capabilities |
|
13620
5b727ab37092
Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents:
13401
diff
changeset
|
1404 at a later point without compromising the APIs. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1405 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1406 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1407 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1408 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1409 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF16( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1410 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1411 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1412 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1413 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1414 ); |
|
73476
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1415 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF16( |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1416 PyObject* unicode, /* Unicode object */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1417 const char *errors, /* error handling */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1418 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1419 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1420 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1421 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1422 /* --- Unicode-Escape Codecs ---------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1423 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1424 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1425 const char *string, /* Unicode-Escape encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1426 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1427 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1428 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1429 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1430 PyAPI_FUNC(PyObject*) PyUnicode_AsUnicodeEscapeString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1431 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1432 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1433 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1434 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1435 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1436 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1437 Py_ssize_t length /* Number of Py_UNICODE chars to encode */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1438 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1439 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1440 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1441 /* --- Raw-Unicode-Escape Codecs ------------------------------------------ */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1442 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1443 PyAPI_FUNC(PyObject*) PyUnicode_DecodeRawUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1444 const char *string, /* Raw-Unicode-Escape encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1445 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1446 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1447 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1448 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1449 PyAPI_FUNC(PyObject*) PyUnicode_AsRawUnicodeEscapeString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1450 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1451 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1452 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1453 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1454 PyAPI_FUNC(PyObject*) PyUnicode_EncodeRawUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1455 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1456 Py_ssize_t length /* Number of Py_UNICODE chars to encode */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1457 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1458 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1459 |
| 35485 | 1460 /* --- Unicode Internal Codec --------------------------------------------- |
| 1461 | |
| 1462 Only for internal use in _codecsmodule.c */ | |
| 1463 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1464 #ifndef Py_LIMITED_API |
| 35485 | 1465 PyObject *_PyUnicode_DecodeUnicodeInternal( |
| 1466 const char *string, | |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1467 Py_ssize_t length, |
| 35485 | 1468 const char *errors |
| 1469 ); | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1470 #endif |
| 35485 | 1471 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1472 /* --- Latin-1 Codecs ----------------------------------------------------- |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1473 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1474 Note: Latin-1 corresponds to the first 256 Unicode ordinals. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1475 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1476 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1477 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1478 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLatin1( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1479 const char *string, /* Latin-1 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1480 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1481 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1482 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1483 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1484 PyAPI_FUNC(PyObject*) PyUnicode_AsLatin1String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1485 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1486 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1487 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1488 #ifndef Py_LIMITED_API |
| 72475 | 1489 PyAPI_FUNC(PyObject*) _PyUnicode_AsLatin1String( |
| 1490 PyObject* unicode, | |
| 1491 const char* errors); | |
| 1492 | |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1493 PyAPI_FUNC(PyObject*) PyUnicode_EncodeLatin1( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1494 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1495 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1496 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1497 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1498 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1499 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1500 /* --- ASCII Codecs ------------------------------------------------------- |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1501 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1502 Only 7-bit ASCII data is excepted. All other codes generate errors. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1503 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1504 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1505 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1506 PyAPI_FUNC(PyObject*) PyUnicode_DecodeASCII( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1507 const char *string, /* ASCII encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1508 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1509 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1510 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1511 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1512 PyAPI_FUNC(PyObject*) PyUnicode_AsASCIIString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1513 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1514 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1515 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1516 #ifndef Py_LIMITED_API |
| 72475 | 1517 PyAPI_FUNC(PyObject*) _PyUnicode_AsASCIIString( |
| 1518 PyObject* unicode, | |
| 1519 const char* errors); | |
| 1520 | |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1521 PyAPI_FUNC(PyObject*) PyUnicode_EncodeASCII( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1522 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1523 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1524 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1525 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1526 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1527 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1528 /* --- Character Map Codecs ----------------------------------------------- |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1529 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1530 This codec uses mappings to encode and decode characters. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1531 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1532 Decoding mappings must map single string characters to single |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1533 Unicode characters, integers (which are then interpreted as Unicode |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1534 ordinals) or None (meaning "undefined mapping" and causing an |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1535 error). |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1536 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1537 Encoding mappings must map single Unicode characters to single |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1538 string characters, integers (which are then interpreted as Latin-1 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1539 ordinals) or None (meaning "undefined mapping" and causing an |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1540 error). |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1541 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1542 If a character lookup fails with a LookupError, the character is |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1543 copied as-is meaning that its ordinal value will be interpreted as |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1544 Unicode or Latin-1 ordinal resp. Because of this mappings only need |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1545 to contain those mappings which map characters to different code |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1546 points. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1547 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1548 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1549 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1550 PyAPI_FUNC(PyObject*) PyUnicode_DecodeCharmap( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1551 const char *string, /* Encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1552 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1553 PyObject *mapping, /* character mapping |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1554 (char ordinal -> unicode ordinal) */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1555 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1556 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1557 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1558 PyAPI_FUNC(PyObject*) PyUnicode_AsCharmapString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1559 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1560 PyObject *mapping /* character mapping |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1561 (unicode ordinal -> char ordinal) */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1562 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1563 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1564 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1565 PyAPI_FUNC(PyObject*) PyUnicode_EncodeCharmap( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1566 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1567 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1568 PyObject *mapping, /* character mapping |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1569 (unicode ordinal -> char ordinal) */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1570 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1571 ); |
|
73302
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1572 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeCharmap( |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1573 PyObject *unicode, /* Unicode object */ |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1574 PyObject *mapping, /* character mapping |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1575 (unicode ordinal -> char ordinal) */ |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1576 const char *errors /* error handling */ |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1577 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1578 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1579 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1580 /* Translate a Py_UNICODE buffer of the given length by applying a |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1581 character mapping table to it and return the resulting Unicode |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1582 object. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1583 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1584 The mapping table must map Unicode ordinal integers to Unicode |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1585 ordinal integers or None (causing deletion of the character). |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1586 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1587 Mapping tables may be dictionaries or sequences. Unmapped character |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1588 ordinals (ones which cause a LookupError) are left untouched and |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1589 are copied as-is. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1590 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1591 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1592 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1593 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1594 PyAPI_FUNC(PyObject *) PyUnicode_TranslateCharmap( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1595 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1596 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1597 PyObject *table, /* Translate table */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1598 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1599 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1600 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1601 |
|
71193
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
1602 #ifdef HAVE_MBCS |
| 12074 | 1603 |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1604 /* --- MBCS codecs for Windows -------------------------------------------- */ |
| 12074 | 1605 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1606 PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCS( |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1607 const char *string, /* MBCS encoded string */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1608 Py_ssize_t length, /* size of string */ |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1609 const char *errors /* error handling */ |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1610 ); |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1611 |
|
39442
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1612 PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCSStateful( |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1613 const char *string, /* MBCS encoded string */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1614 Py_ssize_t length, /* size of string */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1615 const char *errors, /* error handling */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1616 Py_ssize_t *consumed /* bytes consumed */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1617 ); |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1618 |
|
72976
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1619 PyAPI_FUNC(PyObject*) PyUnicode_DecodeCodePageStateful( |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1620 int code_page, /* code page number */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1621 const char *string, /* encoded string */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1622 Py_ssize_t length, /* size of string */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1623 const char *errors, /* error handling */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1624 Py_ssize_t *consumed /* bytes consumed */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1625 ); |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1626 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1627 PyAPI_FUNC(PyObject*) PyUnicode_AsMBCSString( |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1628 PyObject *unicode /* Unicode object */ |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1629 ); |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1630 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1631 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1632 PyAPI_FUNC(PyObject*) PyUnicode_EncodeMBCS( |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1633 const Py_UNICODE *data, /* Unicode char buffer */ |
|
72976
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1634 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1635 const char *errors /* error handling */ |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1636 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1637 #endif |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1638 |
|
72976
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1639 PyAPI_FUNC(PyObject*) PyUnicode_EncodeCodePage( |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1640 int code_page, /* code page number */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1641 PyObject *unicode, /* Unicode object */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1642 const char *errors /* error handling */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1643 ); |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1644 |
|
71193
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
1645 #endif /* HAVE_MBCS */ |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1646 |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1647 /* --- Decimal Encoder ---------------------------------------------------- */ |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1648 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1649 /* Takes a Unicode string holding a decimal value and writes it into |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1650 an output buffer using standard ASCII digit codes. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1651 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1652 The output buffer has to provide at least length+1 bytes of storage |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1653 area. The output string is 0-terminated. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1654 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1655 The encoder converts whitespace to ' ', decimal characters to their |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1656 corresponding ASCII digit and all other Latin-1 characters except |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1657 \0 as-is. Characters outside this range (Unicode ordinals 1-256) |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1658 are treated as errors. This includes embedded NULL bytes. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1659 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1660 Error handling is defined by the errors argument: |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1661 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1662 NULL or "strict": raise a ValueError |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1663 "ignore": ignore the wrong characters (these are not copied to the |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1664 output buffer) |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1665 "replace": replaces illegal characters with '?' |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1666 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1667 Returns 0 on success, -1 on failure. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1668 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1669 */ |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1670 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1671 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1672 PyAPI_FUNC(int) PyUnicode_EncodeDecimal( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1673 Py_UNICODE *s, /* Unicode buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1674 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1675 char *output, /* Output buffer; must have size >= length */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1676 const char *errors /* error handling */ |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1677 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1678 #endif |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1679 |
|
66673
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1680 /* Transforms code points that have decimal digit property to the |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1681 corresponding ASCII digit code points. |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1682 |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1683 Returns a new Unicode string on success, NULL on failure. |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1684 */ |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1685 |
|
66751
902db4722594
Take PyUnicode_TransformDecimalToASCII out of the limited API.
Georg Brandl <georg@python.org>
parents:
66673
diff
changeset
|
1686 #ifndef Py_LIMITED_API |
|
66673
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1687 PyAPI_FUNC(PyObject*) PyUnicode_TransformDecimalToASCII( |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1688 Py_UNICODE *s, /* Unicode buffer */ |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1689 Py_ssize_t length /* Number of Py_UNICODE chars to transform */ |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1690 ); |
|
66751
902db4722594
Take PyUnicode_TransformDecimalToASCII out of the limited API.
Georg Brandl <georg@python.org>
parents:
66673
diff
changeset
|
1691 #endif |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1692 |
|
73591
abe62c9d659d
Fix misused of "PyUnicodeObject" structure name in unicodeobject.h
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73476
diff
changeset
|
1693 /* Similar to PyUnicode_TransformDecimalToASCII(), but takes a PyObject |
| 72475 | 1694 as argument instead of a raw buffer and length. This function additionally |
| 1695 transforms spaces to ASCII because this is what the callers in longobject, | |
| 1696 floatobject, and complexobject did anyways. */ | |
| 1697 | |
| 1698 #ifndef Py_LIMITED_API | |
| 1699 PyAPI_FUNC(PyObject*) _PyUnicode_TransformDecimalAndSpaceToASCII( | |
| 1700 PyObject *unicode /* Unicode object */ | |
| 1701 ); | |
| 1702 #endif | |
| 1703 | |
|
74002
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1704 /* --- Locale encoding --------------------------------------------------- */ |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1705 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1706 /* Decode a string from the current locale encoding. The decoder is strict if |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1707 *surrogateescape* is equal to zero, otherwise it uses the 'surrogateescape' |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1708 error handler (PEP 383) to escape undecodable bytes. If a byte sequence can |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1709 be decoded as a surrogate character and *surrogateescape* is not equal to |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1710 zero, the byte sequence is escaped using the 'surrogateescape' error handler |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1711 instead of being decoded. *str* must end with a null character but cannot |
|
74006
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1712 contain embedded null characters. */ |
|
74002
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1713 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1714 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocaleAndSize( |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1715 const char *str, |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1716 Py_ssize_t len, |
|
74009
07802351ccad
Issue #13560: Locale codec functions use the classic "errors" parameter,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74006
diff
changeset
|
1717 const char *errors); |
|
74002
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1718 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1719 /* Similar to PyUnicode_DecodeLocaleAndSize(), but compute the string |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1720 length using strlen(). */ |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1721 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1722 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocale( |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1723 const char *str, |
|
74009
07802351ccad
Issue #13560: Locale codec functions use the classic "errors" parameter,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74006
diff
changeset
|
1724 const char *errors); |
|
74002
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1725 |
|
74006
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1726 /* Encode a Unicode object to the current locale encoding. The encoder is |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1727 strict is *surrogateescape* is equal to zero, otherwise the |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1728 "surrogateescape" error handler is used. Return a bytes object. The string |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1729 cannot contain embedded null characters.. */ |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1730 |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1731 PyAPI_FUNC(PyObject*) PyUnicode_EncodeLocale( |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1732 PyObject *unicode, |
|
74009
07802351ccad
Issue #13560: Locale codec functions use the classic "errors" parameter,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74006
diff
changeset
|
1733 const char *errors |
|
74006
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1734 ); |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1735 |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1736 /* --- File system encoding ---------------------------------------------- */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1737 |
|
63963
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1738 /* ParseTuple converter: encode str objects to bytes using |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1739 PyUnicode_EncodeFSDefault(); bytes objects are output as-is. */ |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1740 |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1741 PyAPI_FUNC(int) PyUnicode_FSConverter(PyObject*, void*); |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1742 |
|
63963
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1743 /* ParseTuple converter: decode bytes objects to unicode using |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1744 PyUnicode_DecodeFSDefaultAndSize(); str objects are output as-is. */ |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1745 |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1746 PyAPI_FUNC(int) PyUnicode_FSDecoder(PyObject*, void*); |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1747 |
|
61577
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1748 /* Decode a null-terminated string using Py_FileSystemDefaultEncoding |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1749 and the "surrogateescape" error handler. |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1750 |
|
65348
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1751 If Py_FileSystemDefaultEncoding is not set, fall back to the locale |
|
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1752 encoding. |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1753 |
| 61601 | 1754 Use PyUnicode_DecodeFSDefaultAndSize() if the string length is known. |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1755 */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1756 |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1757 PyAPI_FUNC(PyObject*) PyUnicode_DecodeFSDefault( |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1758 const char *s /* encoded string */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1759 ); |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1760 |
|
61577
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1761 /* Decode a string using Py_FileSystemDefaultEncoding |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1762 and the "surrogateescape" error handler. |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1763 |
|
65348
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1764 If Py_FileSystemDefaultEncoding is not set, fall back to the locale |
|
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1765 encoding. |
|
61577
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1766 */ |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1767 |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1768 PyAPI_FUNC(PyObject*) PyUnicode_DecodeFSDefaultAndSize( |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1769 const char *s, /* encoded string */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1770 Py_ssize_t size /* size */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1771 ); |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1772 |
|
61597
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1773 /* Encode a Unicode object to Py_FileSystemDefaultEncoding with the |
| 61601 | 1774 "surrogateescape" error handler, and return bytes. |
|
61597
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1775 |
|
65348
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1776 If Py_FileSystemDefaultEncoding is not set, fall back to the locale |
|
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1777 encoding. |
|
61597
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1778 */ |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1779 |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1780 PyAPI_FUNC(PyObject*) PyUnicode_EncodeFSDefault( |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1781 PyObject *unicode |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1782 ); |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1783 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1784 /* --- Methods & Slots ---------------------------------------------------- |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1785 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1786 These are capable of handling Unicode objects and strings on input |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1787 (we refer to them as strings in the descriptions) and return |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
1788 Unicode objects or integers as appropriate. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1789 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1790 /* Concat two strings giving a new Unicode string. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1791 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1792 PyAPI_FUNC(PyObject*) PyUnicode_Concat( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1793 PyObject *left, /* Left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1794 PyObject *right /* Right string */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1795 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1796 |
|
41769
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1797 /* Concat two strings and put the result in *pleft |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1798 (sets *pleft to NULL on error) */ |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1799 |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1800 PyAPI_FUNC(void) PyUnicode_Append( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1801 PyObject **pleft, /* Pointer to left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1802 PyObject *right /* Right string */ |
|
41769
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1803 ); |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1804 |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1805 /* Concat two strings, put the result in *pleft and drop the right object |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1806 (sets *pleft to NULL on error) */ |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1807 |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1808 PyAPI_FUNC(void) PyUnicode_AppendAndDel( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1809 PyObject **pleft, /* Pointer to left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1810 PyObject *right /* Right string */ |
|
41769
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1811 ); |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1812 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1813 /* Split a string giving a list of Unicode strings. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1814 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1815 If sep is NULL, splitting will be done at all whitespace |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1816 substrings. Otherwise, splits occur at the given separator. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1817 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1818 At most maxsplit splits will be done. If negative, no limit is set. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1819 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1820 Separators are not included in the resulting list. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1821 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1822 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1823 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1824 PyAPI_FUNC(PyObject*) PyUnicode_Split( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1825 PyObject *s, /* String to split */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1826 PyObject *sep, /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1827 Py_ssize_t maxsplit /* Maxsplit count */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1828 ); |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1829 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1830 /* Dito, but split at line breaks. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1831 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1832 CRLF is considered to be one line break. Line breaks are not |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1833 included in the resulting list. */ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1834 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1835 PyAPI_FUNC(PyObject*) PyUnicode_Splitlines( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1836 PyObject *s, /* String to split */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1837 int keepends /* If true, line end markers are included */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1838 ); |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1839 |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1840 /* Partition a string using a given separator. */ |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1841 |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1842 PyAPI_FUNC(PyObject*) PyUnicode_Partition( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1843 PyObject *s, /* String to partition */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1844 PyObject *sep /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1845 ); |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1846 |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1847 /* Partition a string using a given separator, searching from the end of the |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1848 string. */ |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1849 |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1850 PyAPI_FUNC(PyObject*) PyUnicode_RPartition( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1851 PyObject *s, /* String to partition */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1852 PyObject *sep /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1853 ); |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1854 |
|
31199
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1855 /* Split a string giving a list of Unicode strings. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1856 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1857 If sep is NULL, splitting will be done at all whitespace |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1858 substrings. Otherwise, splits occur at the given separator. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1859 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1860 At most maxsplit splits will be done. But unlike PyUnicode_Split |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1861 PyUnicode_RSplit splits from the end of the string. If negative, |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1862 no limit is set. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1863 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1864 Separators are not included in the resulting list. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1865 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1866 */ |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1867 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1868 PyAPI_FUNC(PyObject*) PyUnicode_RSplit( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1869 PyObject *s, /* String to split */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1870 PyObject *sep, /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1871 Py_ssize_t maxsplit /* Maxsplit count */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1872 ); |
|
31199
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1873 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1874 /* Translate a string by applying a character mapping table to it and |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1875 return the resulting Unicode object. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1876 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1877 The mapping table must map Unicode ordinal integers to Unicode |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1878 ordinal integers or None (causing deletion of the character). |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1879 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1880 Mapping tables may be dictionaries or sequences. Unmapped character |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1881 ordinals (ones which cause a LookupError) are left untouched and |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1882 are copied as-is. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1883 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1884 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1885 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1886 PyAPI_FUNC(PyObject *) PyUnicode_Translate( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1887 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1888 PyObject *table, /* Translate table */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1889 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1890 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1891 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1892 /* Join a sequence of strings using the given separator and return |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1893 the resulting Unicode string. */ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1894 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1895 PyAPI_FUNC(PyObject*) PyUnicode_Join( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1896 PyObject *separator, /* Separator string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1897 PyObject *seq /* Sequence object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1898 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1899 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1900 /* Return 1 if substr matches str[start:end] at the given tail end, 0 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1901 otherwise. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1902 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1903 PyAPI_FUNC(Py_ssize_t) PyUnicode_Tailmatch( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1904 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1905 PyObject *substr, /* Prefix or Suffix string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1906 Py_ssize_t start, /* Start index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1907 Py_ssize_t end, /* Stop index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1908 int direction /* Tail end: -1 prefix, +1 suffix */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1909 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1910 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1911 /* Return the first position of substr in str[start:end] using the |
|
23561
c5e8f89fd358
Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents:
23225
diff
changeset
|
1912 given search direction or -1 if not found. -2 is returned in case |
|
c5e8f89fd358
Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents:
23225
diff
changeset
|
1913 an error occurred and an exception is set. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1914 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1915 PyAPI_FUNC(Py_ssize_t) PyUnicode_Find( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1916 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1917 PyObject *substr, /* Substring to find */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1918 Py_ssize_t start, /* Start index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1919 Py_ssize_t end, /* Stop index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1920 int direction /* Find direction: +1 forward, -1 backward */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1921 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1922 |
| 72475 | 1923 /* Like PyUnicode_Find, but search for single character only. */ |
| 1924 PyAPI_FUNC(Py_ssize_t) PyUnicode_FindChar( | |
| 1925 PyObject *str, | |
| 1926 Py_UCS4 ch, | |
| 1927 Py_ssize_t start, | |
| 1928 Py_ssize_t end, | |
| 1929 int direction | |
| 1930 ); | |
| 1931 | |
|
12025
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
1932 /* Count the number of occurrences of substr in str[start:end]. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1933 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1934 PyAPI_FUNC(Py_ssize_t) PyUnicode_Count( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1935 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1936 PyObject *substr, /* Substring to count */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1937 Py_ssize_t start, /* Start index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1938 Py_ssize_t end /* Stop index */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1939 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1940 |
|
12025
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
1941 /* Replace at most maxcount occurrences of substr in str with replstr |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1942 and return the resulting Unicode object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1943 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1944 PyAPI_FUNC(PyObject *) PyUnicode_Replace( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1945 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1946 PyObject *substr, /* Substring to find */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1947 PyObject *replstr, /* Substring to replace */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1948 Py_ssize_t maxcount /* Max. number of replacements to apply; |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1949 -1 = all */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1950 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1951 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1952 /* Compare two strings and return -1, 0, 1 for less than, equal, |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1953 greater than resp. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1954 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1955 PyAPI_FUNC(int) PyUnicode_Compare( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1956 PyObject *left, /* Left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1957 PyObject *right /* Right string */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1958 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1959 |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1960 PyAPI_FUNC(int) PyUnicode_CompareWithASCIIString( |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1961 PyObject *left, |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
1962 const char *right /* ASCII-encoded string */ |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1963 ); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1964 |
|
39610
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1965 /* Rich compare two strings and return one of the following: |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1966 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1967 - NULL in case an exception was raised |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
1968 - Py_True or Py_False for successfully comparisons |
|
39610
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1969 - Py_NotImplemented in case the type combination is unknown |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1970 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1971 Note that Py_EQ and Py_NE comparisons can cause a UnicodeWarning in |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1972 case the conversion of the arguments to Unicode fails with a |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1973 UnicodeDecodeError. |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1974 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1975 Possible values for op: |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1976 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1977 Py_GT, Py_GE, Py_EQ, Py_NE, Py_LT, Py_LE |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1978 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1979 */ |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1980 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1981 PyAPI_FUNC(PyObject *) PyUnicode_RichCompare( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1982 PyObject *left, /* Left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1983 PyObject *right, /* Right string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1984 int op /* Operation: Py_EQ, Py_NE, Py_GT, etc. */ |
|
39610
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1985 ); |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1986 |
|
13620
5b727ab37092
Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents:
13401
diff
changeset
|
1987 /* Apply a argument tuple or dictionary to a format string and return |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1988 the resulting Unicode string. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1989 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1990 PyAPI_FUNC(PyObject *) PyUnicode_Format( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1991 PyObject *format, /* Format string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1992 PyObject *args /* Argument tuple or dictionary */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1993 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1994 |
|
12002
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1995 /* Checks whether element is contained in container and return 1/0 |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1996 accordingly. |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1997 |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1998 element has to coerce to an one element Unicode string. -1 is |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1999 returned in case of an error. */ |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
2000 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2001 PyAPI_FUNC(int) PyUnicode_Contains( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
2002 PyObject *container, /* Container string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
2003 PyObject *element /* Element string */ |
|
12002
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
2004 ); |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
2005 |
|
74680
572bb8c265c0
Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Antoine Pitrou <solipsis@pitrou.net>
parents:
72298
diff
changeset
|
2006 /* Checks whether the string contains any NUL characters. */ |
|
572bb8c265c0
Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Antoine Pitrou <solipsis@pitrou.net>
parents:
72298
diff
changeset
|
2007 |
|
572bb8c265c0
Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Antoine Pitrou <solipsis@pitrou.net>
parents:
72298
diff
changeset
|
2008 #ifndef Py_LIMITED_API |
|
572bb8c265c0
Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Antoine Pitrou <solipsis@pitrou.net>
parents:
72298
diff
changeset
|
2009 PyAPI_FUNC(int) _PyUnicode_HasNULChars(PyObject *); |
|
572bb8c265c0
Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Antoine Pitrou <solipsis@pitrou.net>
parents:
72298
diff
changeset
|
2010 #endif |
|
572bb8c265c0
Issue #13848: open() and the FileIO constructor now check for NUL characters in the file name.
Antoine Pitrou <solipsis@pitrou.net>
parents:
72298
diff
changeset
|
2011 |
|
42597
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
2012 /* Checks whether argument is a valid identifier. */ |
|
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
2013 |
|
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
2014 PyAPI_FUNC(int) PyUnicode_IsIdentifier(PyObject *s); |
|
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
2015 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
2016 #ifndef Py_LIMITED_API |
|
23225
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
2017 /* Externally visible for str.strip(unicode) */ |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2018 PyAPI_FUNC(PyObject *) _PyUnicode_XStrip( |
|
73185
f61db5671cc8
Replace PyUnicodeObject* by PyObject* where it was irrevelant
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72980
diff
changeset
|
2019 PyObject *self, |
|
23225
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
2020 int striptype, |
|
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
2021 PyObject *sepobj |
|
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
2022 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
2023 #endif |
|
23225
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
2024 |
|
52773
28096ed73153
Added ',' thousands grouping to int.__format__. See PEP 378.
Eric Smith <eric@trueblade.com>
parents:
50660
diff
changeset
|
2025 /* Using explicit passed-in values, insert the thousands grouping |
|
28096ed73153
Added ',' thousands grouping to int.__format__. See PEP 378.
Eric Smith <eric@trueblade.com>
parents:
50660
diff
changeset
|
2026 into the string pointed to by buffer. For the argument descriptions, |
|
28096ed73153
Added ',' thousands grouping to int.__format__. See PEP 378.
Eric Smith <eric@trueblade.com>
parents:
50660
diff
changeset
|
2027 see Objects/stringlib/localeutil.h */ |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
2028 #ifndef Py_LIMITED_API |
| 72475 | 2029 PyAPI_FUNC(Py_ssize_t) _PyUnicode_InsertThousandsGrouping( |
|
72705
05ed6e5f2cf4
Add asciilib: similar to ucs1, ucs2 and ucs4 library, but specialized to ASCII
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72699
diff
changeset
|
2030 PyObject *unicode, |
|
75231
f89e2f4cda88
Issue #13706: Fix format(int, "n") for locale with non-ASCII thousands separator
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74702
diff
changeset
|
2031 Py_ssize_t index, |
| 72475 | 2032 Py_ssize_t n_buffer, |
| 2033 void *digits, | |
| 2034 Py_ssize_t n_digits, | |
| 2035 Py_ssize_t min_width, | |
| 2036 const char *grouping, | |
|
75231
f89e2f4cda88
Issue #13706: Fix format(int, "n") for locale with non-ASCII thousands separator
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74702
diff
changeset
|
2037 PyObject *thousands_sep, |
|
f89e2f4cda88
Issue #13706: Fix format(int, "n") for locale with non-ASCII thousands separator
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74702
diff
changeset
|
2038 Py_UCS4 *maxchar); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
2039 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2040 /* === Characters Type APIs =============================================== */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2041 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
2042 /* Helper array used by Py_UNICODE_ISSPACE(). */ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
2043 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
2044 #ifndef Py_LIMITED_API |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
2045 PyAPI_DATA(const unsigned char) _Py_ascii_whitespace[]; |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
2046 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2047 /* These should not be used directly. Use the Py_UNICODE_IS* and |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
2048 Py_UNICODE_TO* macros instead. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2049 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2050 These APIs are implemented in Objects/unicodectype.c. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2051 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2052 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2053 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2054 PyAPI_FUNC(int) _PyUnicode_IsLowercase( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2055 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2056 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2057 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2058 PyAPI_FUNC(int) _PyUnicode_IsUppercase( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2059 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2060 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2061 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2062 PyAPI_FUNC(int) _PyUnicode_IsTitlecase( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2063 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2064 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2065 |
|
42595
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
2066 PyAPI_FUNC(int) _PyUnicode_IsXidStart( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2067 Py_UCS4 ch /* Unicode character */ |
|
42595
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
2068 ); |
|
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
2069 |
|
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
2070 PyAPI_FUNC(int) _PyUnicode_IsXidContinue( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2071 Py_UCS4 ch /* Unicode character */ |
|
42595
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
2072 ); |
|
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
2073 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2074 PyAPI_FUNC(int) _PyUnicode_IsWhitespace( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2075 const Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2076 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2077 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2078 PyAPI_FUNC(int) _PyUnicode_IsLinebreak( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2079 const Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2080 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2081 |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2082 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToLowercase( |
|
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2083 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2084 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2085 |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2086 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToUppercase( |
|
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2087 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2088 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2089 |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2090 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToTitlecase( |
|
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2091 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2092 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2093 |
|
74338
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2094 PyAPI_FUNC(int) _PyUnicode_ToLowerFull( |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2095 Py_UCS4 ch, /* Unicode character */ |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2096 Py_UCS4 *res |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2097 ); |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2098 |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2099 PyAPI_FUNC(int) _PyUnicode_ToTitleFull( |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2100 Py_UCS4 ch, /* Unicode character */ |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2101 Py_UCS4 *res |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2102 ); |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2103 |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2104 PyAPI_FUNC(int) _PyUnicode_ToUpperFull( |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2105 Py_UCS4 ch, /* Unicode character */ |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2106 Py_UCS4 *res |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2107 ); |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2108 |
|
74405
d4669f43d05f
add str.casefold() (closes #13752)
Benjamin Peterson <benjamin@python.org>
parents:
74368
diff
changeset
|
2109 PyAPI_FUNC(int) _PyUnicode_ToFoldedFull( |
|
d4669f43d05f
add str.casefold() (closes #13752)
Benjamin Peterson <benjamin@python.org>
parents:
74368
diff
changeset
|
2110 Py_UCS4 ch, /* Unicode character */ |
|
d4669f43d05f
add str.casefold() (closes #13752)
Benjamin Peterson <benjamin@python.org>
parents:
74368
diff
changeset
|
2111 Py_UCS4 *res |
|
d4669f43d05f
add str.casefold() (closes #13752)
Benjamin Peterson <benjamin@python.org>
parents:
74368
diff
changeset
|
2112 ); |
|
d4669f43d05f
add str.casefold() (closes #13752)
Benjamin Peterson <benjamin@python.org>
parents:
74368
diff
changeset
|
2113 |
|
74338
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2114 PyAPI_FUNC(int) _PyUnicode_IsCaseIgnorable( |
|
74368
8e538ed41766
Silence compilation warnings on Windows
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
74338
diff
changeset
|
2115 Py_UCS4 ch /* Unicode character */ |
|
74338
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2116 ); |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2117 |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2118 PyAPI_FUNC(int) _PyUnicode_IsCased( |
|
74368
8e538ed41766
Silence compilation warnings on Windows
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
74338
diff
changeset
|
2119 Py_UCS4 ch /* Unicode character */ |
|
74338
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2120 ); |
|
f7e05d205a52
use full unicode mappings for upper/lower/title case (#12736)
Benjamin Peterson <benjamin@python.org>
parents:
74261
diff
changeset
|
2121 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2122 PyAPI_FUNC(int) _PyUnicode_ToDecimalDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2123 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2124 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2125 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2126 PyAPI_FUNC(int) _PyUnicode_ToDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2127 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2128 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2129 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2130 PyAPI_FUNC(double) _PyUnicode_ToNumeric( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2131 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2132 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2133 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2134 PyAPI_FUNC(int) _PyUnicode_IsDecimalDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2135 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2136 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2137 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2138 PyAPI_FUNC(int) _PyUnicode_IsDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2139 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2140 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2141 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2142 PyAPI_FUNC(int) _PyUnicode_IsNumeric( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2143 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2144 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2145 |
|
47611
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
2146 PyAPI_FUNC(int) _PyUnicode_IsPrintable( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2147 Py_UCS4 ch /* Unicode character */ |
|
47611
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
2148 ); |
|
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
2149 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2150 PyAPI_FUNC(int) _PyUnicode_IsAlpha( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2151 Py_UCS4 ch /* Unicode character */ |
|
13352
d88535ccaee5
Modified the ISALPHA and ISALNUM macros to use the new lookup APIs
Marc-André Lemburg <mal@egenix.com>
parents:
13320
diff
changeset
|
2152 ); |
|
d88535ccaee5
Modified the ISALPHA and ISALNUM macros to use the new lookup APIs
Marc-André Lemburg <mal@egenix.com>
parents:
13320
diff
changeset
|
2153 |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2154 PyAPI_FUNC(size_t) Py_UNICODE_strlen( |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2155 const Py_UNICODE *u |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2156 ); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2157 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2158 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strcpy( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2159 Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2160 const Py_UNICODE *s2); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2161 |
|
64372
a0d8de04c3a8
Create Py_UNICODE_strcat() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64349
diff
changeset
|
2162 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strcat( |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2163 Py_UNICODE *s1, const Py_UNICODE *s2); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2164 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2165 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strncpy( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2166 Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2167 const Py_UNICODE *s2, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2168 size_t n); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2169 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2170 PyAPI_FUNC(int) Py_UNICODE_strcmp( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2171 const Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2172 const Py_UNICODE *s2 |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2173 ); |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2174 |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2175 PyAPI_FUNC(int) Py_UNICODE_strncmp( |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2176 const Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2177 const Py_UNICODE *s2, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2178 size_t n |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2179 ); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2180 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2181 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strchr( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2182 const Py_UNICODE *s, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2183 Py_UNICODE c |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2184 ); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2185 |
|
63912
a3853bfa2518
Issue #9425: create Py_UNICODE_strrchr() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
62530
diff
changeset
|
2186 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strrchr( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2187 const Py_UNICODE *s, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2188 Py_UNICODE c |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2189 ); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2190 |
|
64373
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2191 /* Create a copy of a unicode string ending with a nul character. Return NULL |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2192 and raise a MemoryError exception on memory allocation failure, otherwise |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2193 return a new allocated buffer (use PyMem_Free() to free the buffer). */ |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2194 |
|
64397
98f337a6957c
Rename PyUnicode_strdup() to PyUnicode_AsUnicodeCopy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64373
diff
changeset
|
2195 PyAPI_FUNC(Py_UNICODE*) PyUnicode_AsUnicodeCopy( |
|
64373
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2196 PyObject *unicode |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2197 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
2198 #endif /* Py_LIMITED_API */ |
|
64373
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2199 |
|
72714
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2200 #if defined(Py_DEBUG) && !defined(Py_LIMITED_API) |
|
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2201 PyAPI_FUNC(int) _PyUnicode_CheckConsistency( |
|
73337
783b4b9dfa5e
Replace PyUnicodeObject type by PyObject
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73302
diff
changeset
|
2202 PyObject *op, |
|
72714
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2203 int check_content); |
|
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2204 #endif |
|
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2205 |
|
72835
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2206 /* Return an interned Unicode object for an Identifier; may fail if there is no memory.*/ |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2207 PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*); |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2208 /* Clear all static strings. */ |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2209 PyAPI_FUNC(void) _PyUnicode_ClearStaticStrings(void); |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2210 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2211 #ifdef __cplusplus |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2212 } |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2213 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2214 #endif /* !Py_UNICODEOBJECT_H */ |