Mercurial > cpython-lru_cache-weakref
annotate Include/unicodeobject.h @ 74006:88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
* Use PyUnicode_EncodeLocale() in time.strftime() if wcsftime() is not
available
* Document my last changes in Misc/NEWS
| author | Victor Stinner <victor.stinner@haypocalc.com> |
|---|---|
| date | Sat, 17 Dec 2011 04:13:41 +0100 |
| parents | 279b0aee0cfb |
| children | 07802351ccad |
| rev | line source |
|---|---|
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1 #ifndef Py_UNICODEOBJECT_H |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2 #define Py_UNICODEOBJECT_H |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
3 |
|
44999
7c02182e2979
Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44586
diff
changeset
|
4 #include <stdarg.h> |
|
7c02182e2979
Merged revisions 60284-60349 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44586
diff
changeset
|
5 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
6 /* |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
7 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
8 Unicode implementation based on original code by Fredrik Lundh, |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
9 modified by Marc-Andre Lemburg (mal@lemburg.com) according to the |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
10 Unicode Integration Proposal. (See |
|
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
11 http://www.egenix.com/files/python/unicode-proposal.txt). |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
12 |
|
13828
254f8b7c4a83
Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents:
13661
diff
changeset
|
13 Copyright (c) Corporation for National Research Initiatives. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
14 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
15 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
16 Original header: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
17 -------------------------------------------------------------------- |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
18 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
19 * Yet another Unicode string type for Python. This type supports the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
20 * 16-bit Basic Multilingual Plane (BMP) only. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
21 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
22 * Written by Fredrik Lundh, January 1999. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
23 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
24 * Copyright (c) 1999 by Secret Labs AB. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
25 * Copyright (c) 1999 by Fredrik Lundh. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
26 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
27 * fredrik@pythonware.com |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
28 * http://www.pythonware.com |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
29 * |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
30 * -------------------------------------------------------------------- |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
31 * This Unicode String Type is |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
32 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
33 * Copyright (c) 1999 by Secret Labs AB |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
34 * Copyright (c) 1999 by Fredrik Lundh |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
35 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
36 * By obtaining, using, and/or copying this software and/or its |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
37 * associated documentation, you agree that you have read, understood, |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
38 * and will comply with the following terms and conditions: |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
39 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
40 * Permission to use, copy, modify, and distribute this software and its |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
41 * associated documentation for any purpose and without fee is hereby |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
42 * granted, provided that the above copyright notice appears in all |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
43 * copies, and that both that copyright notice and this permission notice |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
44 * appear in supporting documentation, and that the name of Secret Labs |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
45 * AB or the author not be used in advertising or publicity pertaining to |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
46 * distribution of the software without specific, written prior |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
47 * permission. |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
48 * |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
49 * SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
50 * THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
51 * FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
52 * ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
53 * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
54 * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
55 * OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
56 * -------------------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
57 |
| 19980 | 58 #include <ctype.h> |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
59 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
60 /* === Internal API ======================================================= */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
61 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
62 /* --- Internal Unicode Format -------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
63 |
|
44586
a2938c55200c
Always define Py_USING_UNICODE, 3rd party software may depend on it. A missing declaration can lead to strange bugs as I had to learn the hard way in the upcoming merge
Christian Heimes <christian@cheimes.de>
parents:
44351
diff
changeset
|
64 /* Python 3.x requires unicode */ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
65 #define Py_USING_UNICODE |
|
44586
a2938c55200c
Always define Py_USING_UNICODE, 3rd party software may depend on it. A missing declaration can lead to strange bugs as I had to learn the hard way in the upcoming merge
Christian Heimes <christian@cheimes.de>
parents:
44351
diff
changeset
|
66 |
| 72475 | 67 #ifndef SIZEOF_WCHAR_T |
| 68 #error Must define SIZEOF_WCHAR_T | |
|
18296
22f85c6a6520
Make Unicode work a bit better on Windows...
Fredrik Lundh <fredrik@pythonware.com>
parents:
18292
diff
changeset
|
69 #endif |
|
22f85c6a6520
Make Unicode work a bit better on Windows...
Fredrik Lundh <fredrik@pythonware.com>
parents:
18292
diff
changeset
|
70 |
| 72475 | 71 #define Py_UNICODE_SIZE SIZEOF_WCHAR_T |
| 72 | |
| 73 /* If wchar_t can be used for UCS-4 storage, set Py_UNICODE_WIDE. | |
| 74 Otherwise, Unicode strings are stored as UCS-2 (with limited support | |
| 75 for UTF-16) */ | |
|
18324
3ba7a358ce02
use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents:
18298
diff
changeset
|
76 |
|
3ba7a358ce02
use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents:
18298
diff
changeset
|
77 #if Py_UNICODE_SIZE >= 4 |
|
3ba7a358ce02
use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents:
18298
diff
changeset
|
78 #define Py_UNICODE_WIDE |
|
18292
ffbb9c918ac0
Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents:
18280
diff
changeset
|
79 #endif |
|
18280
0632dcd69ea3
experimental UCS-4 support: added USE_UCS4_STORAGE define to
Fredrik Lundh <fredrik@pythonware.com>
parents:
17887
diff
changeset
|
80 |
|
64687
9cdba6864eb6
#9210: remove --with-wctype-functions configure option.
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64639
diff
changeset
|
81 /* Set these flags if the platform has "wchar.h" and the |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
82 wchar_t type is a 16-bit unsigned type */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
83 /* #define HAVE_WCHAR_H */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
84 /* #define HAVE_USABLE_WCHAR_T */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
85 |
| 72475 | 86 /* Py_UNICODE was the native Unicode storage format (code unit) used by |
| 87 Python and represents a single Unicode element in the Unicode type. | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
88 With PEP 393, Py_UNICODE is deprecated and replaced with a |
| 72475 | 89 typedef to wchar_t. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
90 |
| 72475 | 91 #ifndef Py_LIMITED_API |
| 92 #define PY_UNICODE_TYPE wchar_t | |
| 93 typedef wchar_t Py_UNICODE; | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
94 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
95 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
96 /* If the compiler provides a wchar_t type we try to support it |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
97 through the interface functions PyUnicode_FromWideChar(), |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
98 PyUnicode_AsWideChar() and PyUnicode_AsWideCharString(). */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
99 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
100 #ifdef HAVE_USABLE_WCHAR_T |
|
13917
03fa40528d17
Added #fndef's to avoid compiler errors.
Marc-André Lemburg <mal@egenix.com>
parents:
13837
diff
changeset
|
101 # ifndef HAVE_WCHAR_H |
|
03fa40528d17
Added #fndef's to avoid compiler errors.
Marc-André Lemburg <mal@egenix.com>
parents:
13837
diff
changeset
|
102 # define HAVE_WCHAR_H |
|
03fa40528d17
Added #fndef's to avoid compiler errors.
Marc-André Lemburg <mal@egenix.com>
parents:
13837
diff
changeset
|
103 # endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
104 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
105 |
| 72475 | 106 #if defined(MS_WINDOWS) |
|
71193
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
107 # define HAVE_MBCS |
|
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
108 #endif |
|
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
109 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
110 #ifdef HAVE_WCHAR_H |
| 12074 | 111 /* Work around a cosmetic bug in BSDI 4.x wchar.h; thanks to Thomas Wouters */ |
| 112 # ifdef _HAVE_BSDI | |
| 113 # include <time.h> | |
| 114 # endif | |
| 19980 | 115 # include <wchar.h> |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
116 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
117 |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
118 /* Py_UCS4 and Py_UCS2 are typedefs for the respective |
| 72475 | 119 unicode representations. */ |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
120 #if SIZEOF_INT == 4 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
121 typedef unsigned int Py_UCS4; |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
122 #elif SIZEOF_LONG == 4 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
123 typedef unsigned long Py_UCS4; |
| 72475 | 124 #else |
| 125 #error "Could not find a proper typedef for Py_UCS4" | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
126 #endif |
|
13393
bcb14627b8f4
Bill Tutt: Added Py_UCS4 typedef to hold UCS4 values (these need
Marc-André Lemburg <mal@egenix.com>
parents:
13352
diff
changeset
|
127 |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
128 #if SIZEOF_SHORT == 2 |
| 72475 | 129 typedef unsigned short Py_UCS2; |
|
73633
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
130 #else |
|
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
131 #error "Could not find a proper typedef for Py_UCS2" |
|
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
132 #endif |
|
1dd575504607
Ensure that Py_UCS4 is 32 bits and Py_UCS2 is 16 bits
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73591
diff
changeset
|
133 |
| 72475 | 134 typedef unsigned char Py_UCS1; |
|
18911
55b45b47fc48
As discussed on python-dev: this patch adds name mangling to
Marc-André Lemburg <mal@egenix.com>
parents:
18887
diff
changeset
|
135 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
136 /* --- Internal Unicode Operations ---------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
137 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
138 /* Since splitting on whitespace is an important use case, and |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
139 whitespace in most situations is solely ASCII whitespace, we |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
140 optimize for the common case by using a quick look-up table |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
141 _Py_ascii_whitespace (see below) with an inlined check. |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
142 |
|
45056
c8fca9b5cd3c
Merged revisions 60408-60440 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44999
diff
changeset
|
143 */ |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
144 #ifndef Py_LIMITED_API |
|
45056
c8fca9b5cd3c
Merged revisions 60408-60440 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
44999
diff
changeset
|
145 #define Py_UNICODE_ISSPACE(ch) \ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
146 ((ch) < 128U ? _Py_ascii_whitespace[(ch)] : _PyUnicode_IsWhitespace(ch)) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
147 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
148 #define Py_UNICODE_ISLOWER(ch) _PyUnicode_IsLowercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
149 #define Py_UNICODE_ISUPPER(ch) _PyUnicode_IsUppercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
150 #define Py_UNICODE_ISTITLE(ch) _PyUnicode_IsTitlecase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
151 #define Py_UNICODE_ISLINEBREAK(ch) _PyUnicode_IsLinebreak(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
152 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
153 #define Py_UNICODE_TOLOWER(ch) _PyUnicode_ToLowercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
154 #define Py_UNICODE_TOUPPER(ch) _PyUnicode_ToUppercase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
155 #define Py_UNICODE_TOTITLE(ch) _PyUnicode_ToTitlecase(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
156 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
157 #define Py_UNICODE_ISDECIMAL(ch) _PyUnicode_IsDecimalDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
158 #define Py_UNICODE_ISDIGIT(ch) _PyUnicode_IsDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
159 #define Py_UNICODE_ISNUMERIC(ch) _PyUnicode_IsNumeric(ch) |
|
47611
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
160 #define Py_UNICODE_ISPRINTABLE(ch) _PyUnicode_IsPrintable(ch) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
161 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
162 #define Py_UNICODE_TODECIMAL(ch) _PyUnicode_ToDecimalDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
163 #define Py_UNICODE_TODIGIT(ch) _PyUnicode_ToDigit(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
164 #define Py_UNICODE_TONUMERIC(ch) _PyUnicode_ToNumeric(ch) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
165 |
|
13352
d88535ccaee5
Modified the ISALPHA and ISALNUM macros to use the new lookup APIs
Marc-André Lemburg <mal@egenix.com>
parents:
13320
diff
changeset
|
166 #define Py_UNICODE_ISALPHA(ch) _PyUnicode_IsAlpha(ch) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
167 |
|
13320
67a04816ea50
Added new Py_UNICODE_ISALPHA() and Py_UNICODE_ISALNUM() macros
Marc-André Lemburg <mal@egenix.com>
parents:
12956
diff
changeset
|
168 #define Py_UNICODE_ISALNUM(ch) \ |
|
67a04816ea50
Added new Py_UNICODE_ISALPHA() and Py_UNICODE_ISALNUM() macros
Marc-André Lemburg <mal@egenix.com>
parents:
12956
diff
changeset
|
169 (Py_UNICODE_ISALPHA(ch) || \ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
170 Py_UNICODE_ISDECIMAL(ch) || \ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
171 Py_UNICODE_ISDIGIT(ch) || \ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
172 Py_UNICODE_ISNUMERIC(ch)) |
|
13320
67a04816ea50
Added new Py_UNICODE_ISALPHA() and Py_UNICODE_ISALNUM() macros
Marc-André Lemburg <mal@egenix.com>
parents:
12956
diff
changeset
|
173 |
| 72475 | 174 #define Py_UNICODE_COPY(target, source, length) \ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
175 Py_MEMCPY((target), (source), (length)*sizeof(Py_UNICODE)) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
176 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
177 #define Py_UNICODE_FILL(target, value, length) \ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
178 do {Py_ssize_t i_; Py_UNICODE *t_ = (target); Py_UNICODE v_ = (value);\ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
179 for (i_ = 0; i_ < (length); i_++) t_[i_] = v_;\ |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
180 } while (0) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
181 |
|
72036
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
182 /* macros to work with surrogates */ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
183 #define Py_UNICODE_IS_SURROGATE(ch) (0xD800 <= ch && ch <= 0xDFFF) |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
184 #define Py_UNICODE_IS_HIGH_SURROGATE(ch) (0xD800 <= ch && ch <= 0xDBFF) |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
185 #define Py_UNICODE_IS_LOW_SURROGATE(ch) (0xDC00 <= ch && ch <= 0xDFFF) |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
186 /* Join two surrogate characters and return a single Py_UCS4 value. */ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
187 #define Py_UNICODE_JOIN_SURROGATES(high, low) \ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
188 (((((Py_UCS4)(high) & 0x03FF) << 10) | \ |
|
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
189 ((Py_UCS4)(low) & 0x03FF)) + 0x10000) |
|
73793
351cdf87bed0
Py_UNICODE_HIGH_SURROGATE() and Py_UNICODE_LOW_SURROGATE() macros
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73654
diff
changeset
|
190 /* high surrogate = top 10 bits added to D800 */ |
|
351cdf87bed0
Py_UNICODE_HIGH_SURROGATE() and Py_UNICODE_LOW_SURROGATE() macros
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73654
diff
changeset
|
191 #define Py_UNICODE_HIGH_SURROGATE(ch) (0xD800 | (((ch) - 0x10000) >> 10)) |
|
351cdf87bed0
Py_UNICODE_HIGH_SURROGATE() and Py_UNICODE_LOW_SURROGATE() macros
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73654
diff
changeset
|
192 /* low surrogate = bottom 10 bits added to DC00 */ |
|
351cdf87bed0
Py_UNICODE_HIGH_SURROGATE() and Py_UNICODE_LOW_SURROGATE() macros
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73654
diff
changeset
|
193 #define Py_UNICODE_LOW_SURROGATE(ch) (0xDC00 | (((ch) - 0x10000) & 0x3FF)) |
|
72036
77171f993bf2
#10542: Add 4 macros to work with surrogates: Py_UNICODE_IS_SURROGATE, Py_UNICODE_IS_HIGH_SURROGATE, Py_UNICODE_IS_LOW_SURROGATE, Py_UNICODE_JOIN_SURROGATES.
Ezio Melotti <ezio.melotti@gmail.com>
parents:
71193
diff
changeset
|
194 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
195 /* Check if substring matches at given offset. The offset must be |
|
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
196 valid, and the substring must not be empty. */ |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
197 |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
198 #define Py_UNICODE_MATCH(string, offset, substring) \ |
| 72475 | 199 ((*((string)->wstr + (offset)) == *((substring)->wstr)) && \ |
| 200 ((*((string)->wstr + (offset) + (substring)->wstr_length-1) == *((substring)->wstr + (substring)->wstr_length-1))) && \ | |
| 201 !memcmp((string)->wstr + (offset), (substring)->wstr, (substring)->wstr_length*sizeof(Py_UNICODE))) | |
| 202 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
203 #endif /* Py_LIMITED_API */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
204 |
|
12025
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
205 #ifdef __cplusplus |
|
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
206 extern "C" { |
|
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
207 #endif |
|
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
208 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
209 /* --- Unicode Type ------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
210 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
211 #ifndef Py_LIMITED_API |
| 72475 | 212 |
| 213 /* ASCII-only strings created through PyUnicode_New use the PyASCIIObject | |
| 214 structure. state.ascii and state.compact are set, and the data | |
| 215 immediately follow the structure. utf8_length and wstr_length can be found | |
| 216 in the length field; the utf8 pointer is equal to the data pointer. */ | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
217 typedef struct { |
| 72729 | 218 /* There are 4 forms of Unicode strings: |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
219 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
220 - compact ascii: |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
221 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
222 * structure = PyASCIIObject |
|
73941
0eea55925d1e
resize_copy() now supports legacy ready strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73939
diff
changeset
|
223 * test: PyUnicode_IS_COMPACT_ASCII(op) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
224 * kind = PyUnicode_1BYTE_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
225 * compact = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
226 * ascii = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
227 * ready = 1 |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
228 * (length is the length of the utf8 and wstr strings) |
|
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
229 * (data starts just after the structure) |
|
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
230 * (since ASCII is decoded from UTF-8, the utf8 string are the data) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
231 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
232 - compact: |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
233 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
234 * structure = PyCompactUnicodeObject |
|
73941
0eea55925d1e
resize_copy() now supports legacy ready strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73939
diff
changeset
|
235 * test: PyUnicode_IS_ASCII(op) && !PyUnicode_IS_COMPACT(op) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
236 * kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
237 PyUnicode_4BYTE_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
238 * compact = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
239 * ready = 1 |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
240 * ascii = 0 |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
241 * utf8 is not shared with data |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
242 * utf8_length = 0 if utf8 is NULL |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
243 * wstr is shared with data and wstr_length=length |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
244 if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2 |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
245 or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_t)=4 |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
246 * wstr_length = 0 if wstr is NULL |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
247 * (data starts just after the structure) |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
248 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
249 - legacy string, not ready: |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
250 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
251 * structure = PyUnicodeObject |
|
73941
0eea55925d1e
resize_copy() now supports legacy ready strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73939
diff
changeset
|
252 * test: kind == PyUnicode_WCHAR_KIND |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
253 * length = 0 (use wstr_length) |
|
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
254 * hash = -1 |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
255 * kind = PyUnicode_WCHAR_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
256 * compact = 0 |
|
72632
9c9ebb07d053
Complete documentation of compact ASCII strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72629
diff
changeset
|
257 * ascii = 0 |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
258 * ready = 0 |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
259 * interned = SSTATE_NOT_INTERNED |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
260 * wstr is not NULL |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
261 * data.any is NULL |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
262 * utf8 is NULL |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
263 * utf8_length = 0 |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
264 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
265 - legacy string, ready: |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
266 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
267 * structure = PyUnicodeObject structure |
|
73941
0eea55925d1e
resize_copy() now supports legacy ready strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73939
diff
changeset
|
268 * test: !PyUnicode_IS_COMPACT(op) && kind != PyUnicode_WCHAR_KIND |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
269 * kind = PyUnicode_1BYTE_KIND, PyUnicode_2BYTE_KIND or |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
270 PyUnicode_4BYTE_KIND |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
271 * compact = 0 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
272 * ready = 1 |
|
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
273 * data.any is not NULL |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
274 * utf8 is shared and utf8_length = length with data.any if ascii = 1 |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
275 * utf8_length = 0 if utf8 is NULL |
|
73356
b3543ef9b445
Fix gdb/libpython.py for not ready Unicode strings
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73337
diff
changeset
|
276 * wstr is shared with data.any and wstr_length = length |
|
72629
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
277 if kind=PyUnicode_2BYTE_KIND and sizeof(wchar_t)=2 |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
278 or if kind=PyUnicode_4BYTE_KIND and sizeof(wchar_4)=4 |
|
5346409167a7
Document utf8_length and wstr_length states
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72625
diff
changeset
|
279 * wstr_length = 0 if wstr is NULL |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
280 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
281 Compact strings use only one memory block (structure + characters), |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
282 whereas legacy strings use one block for the structure and one block |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
283 for characters. |
|
72603
dbb9313c3ed8
Add _PyUnicode_CheckConsistency() macro to help debugging
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72598
diff
changeset
|
284 |
|
72621
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
285 Legacy strings are created by PyUnicode_FromUnicode() and |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
286 PyUnicode_FromStringAndSize(NULL, size) functions. They become ready |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
287 when PyUnicode_READY() is called. |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
288 |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
289 See also _PyUnicode_CheckConsistency(). |
|
fe10f0bcc860
Improve string forms and PyUnicode_Resize() documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72617
diff
changeset
|
290 */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
291 PyObject_HEAD |
| 72475 | 292 Py_ssize_t length; /* Number of code points in the string */ |
|
65494
e8969fb822fc
make hashes always the size of pointers; introduce Py_hash_t #9778
Benjamin Peterson <benjamin@python.org>
parents:
65348
diff
changeset
|
293 Py_hash_t hash; /* Hash value; -1 if not set */ |
| 72475 | 294 struct { |
| 295 /* | |
| 296 SSTATE_NOT_INTERNED (0) | |
| 297 SSTATE_INTERNED_MORTAL (1) | |
| 298 SSTATE_INTERNED_IMMORTAL (2) | |
| 299 | |
| 300 If interned != SSTATE_NOT_INTERNED, the two references from the | |
| 301 dictionary to this object are *not* counted in ob_refcnt. | |
| 302 */ | |
| 303 unsigned int interned:2; | |
| 304 /* Character size: | |
| 305 | |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
306 - PyUnicode_WCHAR_KIND (0): |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
307 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
308 * character type = wchar_t (16 or 32 bits, depending on the |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
309 platform) |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
310 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
311 - PyUnicode_1BYTE_KIND (1): |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
312 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
313 * character type = Py_UCS1 (8 bits, unsigned) |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
314 * all characters are in the range U+0000-U+00FF (latin1) |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
315 * if ascii is set, all characters are in the range U+0000-U+007F |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
316 (ASCII), otherwise at least one character is in the range |
|
72715
341c3002ffb2
rephrase PyUnicode_1BYTE_KIND documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72714
diff
changeset
|
317 U+0080-U+00FF |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
318 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
319 - PyUnicode_2BYTE_KIND (2): |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
320 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
321 * character type = Py_UCS2 (16 bits, unsigned) |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
322 * all characters are in the range U+0000-U+FFFF (BMP) |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
323 * at least one character is in the range U+0100-U+FFFF |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
324 |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
325 - PyUnicode_4BYTE_KIND (4): |
|
72699
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
326 |
|
055174308822
Document requierements of Unicode kinds
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72691
diff
changeset
|
327 * character type = Py_UCS4 (32 bits, unsigned) |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
328 * all characters are in the range U+0000-U+10FFFF |
|
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
329 * at least one character is in the range U+10000-U+10FFFF |
| 72475 | 330 */ |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
331 unsigned int kind:3; |
| 72475 | 332 /* Compact is with respect to the allocation scheme. Compact unicode |
| 333 objects only require one memory block while non-compact objects use | |
| 334 one block for the PyUnicodeObject struct and another for its data | |
| 335 buffer. */ | |
| 336 unsigned int compact:1; | |
|
73638
26af48f65ef3
_PyUnicode_CheckConsistency() also checks maxchar maximum value,
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73635
diff
changeset
|
337 /* The string only contains characters in the range U+0000-U+007F (ASCII) |
|
72715
341c3002ffb2
rephrase PyUnicode_1BYTE_KIND documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72714
diff
changeset
|
338 and the kind is PyUnicode_1BYTE_KIND. If ascii is set and compact is |
|
341c3002ffb2
rephrase PyUnicode_1BYTE_KIND documentation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72714
diff
changeset
|
339 set, use the PyASCIIObject structure. */ |
| 72475 | 340 unsigned int ascii:1; |
| 341 /* The ready flag indicates whether the object layout is initialized | |
| 342 completely. This means that this is either a compact object, or | |
| 343 the data pointer is filled out. The bit is redundant, and helps | |
| 344 to minimize the test in PyUnicode_IS_READY(). */ | |
| 345 unsigned int ready:1; | |
| 346 } state; | |
| 347 wchar_t *wstr; /* wchar_t representation (null-terminated) */ | |
| 348 } PyASCIIObject; | |
| 349 | |
| 350 /* Non-ASCII strings allocated through PyUnicode_New use the | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
351 PyCompactUnicodeObject structure. state.compact is set, and the data |
| 72475 | 352 immediately follow the structure. */ |
| 353 typedef struct { | |
| 354 PyASCIIObject _base; | |
| 355 Py_ssize_t utf8_length; /* Number of bytes in utf8, excluding the | |
| 356 * terminating \0. */ | |
| 357 char *utf8; /* UTF-8 representation (null-terminated) */ | |
| 358 Py_ssize_t wstr_length; /* Number of code points in wstr, possible | |
| 359 * surrogates count as two code points. */ | |
| 360 } PyCompactUnicodeObject; | |
| 361 | |
| 362 /* Strings allocated through PyUnicode_FromUnicode(NULL, len) use the | |
| 363 PyUnicodeObject structure. The actual string data is initially in the wstr | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
364 block, and copied into the data block using _PyUnicode_Ready. */ |
| 72475 | 365 typedef struct { |
| 366 PyCompactUnicodeObject _base; | |
| 367 union { | |
| 368 void *any; | |
| 369 Py_UCS1 *latin1; | |
| 370 Py_UCS2 *ucs2; | |
| 371 Py_UCS4 *ucs4; | |
| 372 } data; /* Canonical, smallest-form Unicode buffer */ | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
373 } PyUnicodeObject; |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
374 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
375 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
376 PyAPI_DATA(PyTypeObject) PyUnicode_Type; |
|
44061
0cc552ab76bd
Added all PyTypeObjects to the appropriate header files.
Christian Heimes <christian@cheimes.de>
parents:
43933
diff
changeset
|
377 PyAPI_DATA(PyTypeObject) PyUnicodeIter_Type; |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
378 |
|
40909
f2f0b41f270d
Merged revisions 53875-53911 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
39610
diff
changeset
|
379 #define PyUnicode_Check(op) \ |
|
44351
86a2679dca46
#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT.
Christian Heimes <christian@cheimes.de>
parents:
44061
diff
changeset
|
380 PyType_FastSubclass(Py_TYPE(op), Py_TPFLAGS_UNICODE_SUBCLASS) |
|
86a2679dca46
#1629: Renamed Py_Size, Py_Type and Py_Refcnt to Py_SIZE, Py_TYPE and Py_REFCNT.
Christian Heimes <christian@cheimes.de>
parents:
44061
diff
changeset
|
381 #define PyUnicode_CheckExact(op) (Py_TYPE(op) == &PyUnicode_Type) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
382 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
383 /* Fast access macros */ |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
384 #ifndef Py_LIMITED_API |
| 72475 | 385 |
| 386 #define PyUnicode_WSTR_LENGTH(op) \ | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
387 (PyUnicode_IS_COMPACT_ASCII(op) ? \ |
| 72475 | 388 ((PyASCIIObject*)op)->length : \ |
| 389 ((PyCompactUnicodeObject*)op)->wstr_length) | |
| 390 | |
| 391 /* Returns the deprecated Py_UNICODE representation's size in code units | |
| 392 (this includes surrogate pairs as 2 units). | |
| 393 If the Py_UNICODE representation is not available, it will be computed | |
| 394 on request. Use PyUnicode_GET_LENGTH() for the length in code points. */ | |
| 395 | |
|
73654
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
396 #define PyUnicode_GET_SIZE(op) \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
397 (assert(PyUnicode_Check(op)), \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
398 (((PyASCIIObject *)(op))->wstr) ? \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
399 PyUnicode_WSTR_LENGTH(op) : \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
400 ((void)PyUnicode_AsUnicode((PyObject *)(op)), \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
401 assert(((PyASCIIObject *)(op))->wstr), \ |
|
98b8849d1a30
PyUnicode_GET_SIZE() checks that PyUnicode_AsUnicode() succeed
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73638
diff
changeset
|
402 PyUnicode_WSTR_LENGTH(op))) |
| 72475 | 403 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
404 #define PyUnicode_GET_DATA_SIZE(op) \ |
| 72475 | 405 (PyUnicode_GET_SIZE(op) * Py_UNICODE_SIZE) |
| 406 | |
| 407 /* Alias for PyUnicode_AsUnicode(). This will create a wchar_t/Py_UNICODE | |
| 408 representation on demand. Using this macro is very inefficient now, | |
| 409 try to port your code to use the new PyUnicode_*BYTE_DATA() macros or | |
| 410 use PyUnicode_WRITE() and PyUnicode_READ(). */ | |
| 411 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
412 #define PyUnicode_AS_UNICODE(op) \ |
| 72475 | 413 (assert(PyUnicode_Check(op)), \ |
| 414 (((PyASCIIObject *)(op))->wstr) ? (((PyASCIIObject *)(op))->wstr) : \ | |
| 415 PyUnicode_AsUnicode((PyObject *)(op))) | |
| 416 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
417 #define PyUnicode_AS_DATA(op) \ |
| 72475 | 418 ((const char *)(PyUnicode_AS_UNICODE(op))) |
| 419 | |
| 420 | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
421 /* --- Flexible String Representation Helper Macros (PEP 393) -------------- */ |
| 72475 | 422 |
|
73591
abe62c9d659d
Fix misused of "PyUnicodeObject" structure name in unicodeobject.h
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73476
diff
changeset
|
423 /* Values for PyASCIIObject.state: */ |
| 72475 | 424 |
| 425 /* Interning state. */ | |
| 426 #define SSTATE_NOT_INTERNED 0 | |
| 427 #define SSTATE_INTERNED_MORTAL 1 | |
| 428 #define SSTATE_INTERNED_IMMORTAL 2 | |
| 429 | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
430 /* Return true if the string contains only ASCII characters, or 0 if not. The |
|
73939
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
431 string may be compact (PyUnicode_IS_COMPACT_ASCII) or not, but must be |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
432 ready. */ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
433 #define PyUnicode_IS_ASCII(op) \ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
434 (assert(PyUnicode_Check(op)), \ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
435 assert(PyUnicode_IS_READY(op)), \ |
|
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
436 ((PyASCIIObject*)op)->state.ascii) |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
437 |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
438 /* Return true if the string is compact or 0 if not. |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
439 No type checks or Ready calls are performed. */ |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
440 #define PyUnicode_IS_COMPACT(op) \ |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
441 (((PyASCIIObject*)(op))->state.compact) |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
442 |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
443 /* Return true if the string is a compact ASCII string (use PyASCIIObject |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
444 structure), or 0 if not. No type checks or Ready calls are performed. */ |
|
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
445 #define PyUnicode_IS_COMPACT_ASCII(op) \ |
|
73939
d36c4ec5342b
PyUnicode_IS_ASCII() macro ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73793
diff
changeset
|
446 (((PyASCIIObject*)op)->state.ascii && PyUnicode_IS_COMPACT(op)) |
| 72475 | 447 |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
448 enum PyUnicode_Kind { |
| 72475 | 449 /* String contains only wstr byte characters. This is only possible |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
450 when the string was created with a legacy API and _PyUnicode_Ready() |
| 72475 | 451 has not been called yet. */ |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
452 PyUnicode_WCHAR_KIND = 0, |
| 72475 | 453 /* Return values of the PyUnicode_KIND() macro: */ |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
454 PyUnicode_1BYTE_KIND = 1, |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
455 PyUnicode_2BYTE_KIND = 2, |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
456 PyUnicode_4BYTE_KIND = 4 |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
457 }; |
| 72475 | 458 |
| 72688 | 459 /* Return pointers to the canonical representation cast to unsigned char, |
| 72475 | 460 Py_UCS2, or Py_UCS4 for direct character access. |
|
72796
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
461 No checks are performed, use PyUnicode_KIND() before to ensure |
|
c25262e97304
Change PyUnicode_KIND to 1,2,4. Drop _KIND_SIZE and _CHARACTER_SIZE.
Martin v. Löwis <martin@v.loewis.de>
parents:
72785
diff
changeset
|
462 these will work correctly. */ |
| 72475 | 463 |
| 464 #define PyUnicode_1BYTE_DATA(op) ((Py_UCS1*)PyUnicode_DATA(op)) | |
| 465 #define PyUnicode_2BYTE_DATA(op) ((Py_UCS2*)PyUnicode_DATA(op)) | |
| 466 #define PyUnicode_4BYTE_DATA(op) ((Py_UCS4*)PyUnicode_DATA(op)) | |
| 467 | |
|
72494
93cc591cffe4
Strip trailing spaces in unicodeobject.[ch]
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72492
diff
changeset
|
468 /* Return one of the PyUnicode_*_KIND values defined above. */ |
| 72475 | 469 #define PyUnicode_KIND(op) \ |
| 470 (assert(PyUnicode_Check(op)), \ | |
| 471 assert(PyUnicode_IS_READY(op)), \ | |
| 472 ((PyASCIIObject *)(op))->state.kind) | |
| 473 | |
|
72494
93cc591cffe4
Strip trailing spaces in unicodeobject.[ch]
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72492
diff
changeset
|
474 /* Return a void pointer to the raw unicode buffer. */ |
| 72475 | 475 #define _PyUnicode_COMPACT_DATA(op) \ |
|
72980
cade7c20c728
Simplify _PyUnicode_COMPACT_DATA() macro
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72976
diff
changeset
|
476 (PyUnicode_IS_ASCII(op) ? \ |
| 72475 | 477 ((void*)((PyASCIIObject*)(op) + 1)) : \ |
| 478 ((void*)((PyCompactUnicodeObject*)(op) + 1))) | |
| 479 | |
| 480 #define _PyUnicode_NONCOMPACT_DATA(op) \ | |
| 481 (assert(((PyUnicodeObject*)(op))->data.any), \ | |
| 482 ((((PyUnicodeObject *)(op))->data.any))) | |
| 483 | |
| 484 #define PyUnicode_DATA(op) \ | |
| 485 (assert(PyUnicode_Check(op)), \ | |
| 486 PyUnicode_IS_COMPACT(op) ? _PyUnicode_COMPACT_DATA(op) : \ | |
| 487 _PyUnicode_NONCOMPACT_DATA(op)) | |
| 488 | |
| 489 /* In the access macros below, "kind" may be evaluated more than once. | |
| 490 All other macro parameters are evaluated exactly once, so it is safe | |
| 491 to put side effects into them (such as increasing the index). */ | |
| 492 | |
| 493 /* Write into the canonical representation, this macro does not do any sanity | |
| 494 checks and is intended for usage in loops. The caller should cache the | |
| 72691 | 495 kind and data pointers obtained from other macro calls. |
| 72475 | 496 index is the index in the string (starts at 0) and value is the new |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
497 code point value which should be written to that location. */ |
| 72475 | 498 #define PyUnicode_WRITE(kind, data, index, value) \ |
| 499 do { \ | |
| 500 switch ((kind)) { \ | |
| 501 case PyUnicode_1BYTE_KIND: { \ | |
| 502 ((Py_UCS1 *)(data))[(index)] = (Py_UCS1)(value); \ | |
| 503 break; \ | |
| 504 } \ | |
| 505 case PyUnicode_2BYTE_KIND: { \ | |
| 506 ((Py_UCS2 *)(data))[(index)] = (Py_UCS2)(value); \ | |
| 507 break; \ | |
| 508 } \ | |
| 509 default: { \ | |
| 510 assert((kind) == PyUnicode_4BYTE_KIND); \ | |
| 511 ((Py_UCS4 *)(data))[(index)] = (Py_UCS4)(value); \ | |
| 512 } \ | |
| 513 } \ | |
| 514 } while (0) | |
| 515 | |
| 72691 | 516 /* Read a code point from the string's canonical representation. No checks |
| 72475 | 517 or ready calls are performed. */ |
| 518 #define PyUnicode_READ(kind, data, index) \ | |
| 519 ((Py_UCS4) \ | |
| 520 ((kind) == PyUnicode_1BYTE_KIND ? \ | |
|
72577
5231de1080b0
Use Py_UCS1 instead of unsigned char in unicodeobject.h
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72575
diff
changeset
|
521 ((const Py_UCS1 *)(data))[(index)] : \ |
| 72475 | 522 ((kind) == PyUnicode_2BYTE_KIND ? \ |
| 523 ((const Py_UCS2 *)(data))[(index)] : \ | |
| 524 ((const Py_UCS4 *)(data))[(index)] \ | |
| 525 ) \ | |
| 526 )) | |
| 527 | |
| 528 /* PyUnicode_READ_CHAR() is less efficient than PyUnicode_READ() because it | |
| 529 calls PyUnicode_KIND() and might call it twice. For single reads, use | |
| 530 PyUnicode_READ_CHAR, for multiple consecutive reads callers should | |
| 531 cache kind and use PyUnicode_READ instead. */ | |
| 532 #define PyUnicode_READ_CHAR(unicode, index) \ | |
|
72598
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
533 (assert(PyUnicode_Check(unicode)), \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
534 assert(PyUnicode_IS_READY(unicode)), \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
535 (Py_UCS4) \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
536 (PyUnicode_KIND((unicode)) == PyUnicode_1BYTE_KIND ? \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
537 ((const Py_UCS1 *)(PyUnicode_DATA((unicode))))[(index)] : \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
538 (PyUnicode_KIND((unicode)) == PyUnicode_2BYTE_KIND ? \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
539 ((const Py_UCS2 *)(PyUnicode_DATA((unicode))))[(index)] : \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
540 ((const Py_UCS4 *)(PyUnicode_DATA((unicode))))[(index)] \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
541 ) \ |
|
3a0af974f1b5
PyUnicode_READ_CHAR() ensures that the string is ready
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72577
diff
changeset
|
542 )) |
| 72475 | 543 |
| 544 /* Returns the length of the unicode string. The caller has to make sure that | |
| 545 the string has it's canonical representation set before calling | |
| 546 this macro. Call PyUnicode_(FAST_)Ready to ensure that. */ | |
| 547 #define PyUnicode_GET_LENGTH(op) \ | |
| 548 (assert(PyUnicode_Check(op)), \ | |
| 549 assert(PyUnicode_IS_READY(op)), \ | |
| 550 ((PyASCIIObject *)(op))->length) | |
| 551 | |
| 552 | |
| 553 /* Fast check to determine whether an object is ready. Equivalent to | |
| 554 PyUnicode_IS_COMPACT(op) || ((PyUnicodeObject*)(op))->data.any) */ | |
| 555 | |
| 556 #define PyUnicode_IS_READY(op) (((PyASCIIObject*)op)->state.ready) | |
| 557 | |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
558 /* PyUnicode_READY() does less work than _PyUnicode_Ready() in the best |
| 72475 | 559 case. If the canonical representation is not yet set, it will still call |
|
72614
93f033c7cfea
PyUnicode_Ready() now sets ascii=1 if maxchar < 128
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72603
diff
changeset
|
560 _PyUnicode_Ready(). |
| 72475 | 561 Returns 0 on success and -1 on errors. */ |
| 562 #define PyUnicode_READY(op) \ | |
| 563 (assert(PyUnicode_Check(op)), \ | |
| 564 (PyUnicode_IS_READY(op) ? \ | |
|
72529
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
565 0 : _PyUnicode_Ready((PyObject *)(op)))) |
| 72475 | 566 |
| 567 /* Return a maximum character value which is suitable for creating another | |
| 568 string based on op. This is always an approximation but more efficient | |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
569 than iterating over the string. */ |
| 72475 | 570 #define PyUnicode_MAX_CHAR_VALUE(op) \ |
| 571 (assert(PyUnicode_IS_READY(op)), \ | |
|
72904
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
572 (PyUnicode_IS_ASCII(op) ? \ |
|
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
573 (0x7f) : \ |
| 72475 | 574 (PyUnicode_KIND(op) == PyUnicode_1BYTE_KIND ? \ |
|
72904
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
575 (0xffU) : \ |
| 72475 | 576 (PyUnicode_KIND(op) == PyUnicode_2BYTE_KIND ? \ |
|
72904
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
577 (0xffffU) : \ |
|
f60e8228aeb2
Simplify PyUnicode_MAX_CHAR_VALUE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72836
diff
changeset
|
578 (0x10ffffU))))) |
| 72475 | 579 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
580 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
581 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
582 /* --- Constants ---------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
583 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
584 /* This Unicode character will be used as replacement character during |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
585 decoding if the errors argument is set to "replace". Note: the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
586 Unicode character U+FFFD is the official REPLACEMENT CHARACTER in |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
587 Unicode 3.0. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
588 |
|
72488
17626ba912fb
Set Py_UNICODE_REPLACEMENT_CHARACTER type to Py_UCS4, instead of Py_UNICODE
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72475
diff
changeset
|
589 #define Py_UNICODE_REPLACEMENT_CHARACTER ((Py_UCS4) 0xFFFD) |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
590 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
591 /* === Public API ========================================================= */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
592 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
593 /* --- Plain Py_UNICODE --------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
594 |
| 72475 | 595 /* With PEP 393, this is the recommended way to allocate a new unicode object. |
| 596 This function will allocate the object and its buffer in a single memory | |
| 597 block. Objects created using this function are not resizable. */ | |
| 598 #ifndef Py_LIMITED_API | |
| 599 PyAPI_FUNC(PyObject*) PyUnicode_New( | |
| 600 Py_ssize_t size, /* Number of code points in the new string */ | |
| 601 Py_UCS4 maxchar /* maximum code point value in the string */ | |
| 602 ); | |
| 603 #endif | |
| 604 | |
|
72529
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
605 /* Initializes the canonical string representation from a the deprecated |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
606 wstr/Py_UNICODE representation. This function is used to convert Unicode |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
607 objects which were created using the old API to the new flexible format |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
608 introduced with PEP 393. |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
609 |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
610 Don't call this function directly, use the public PyUnicode_READY() macro |
|
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
611 instead. */ |
| 72475 | 612 #ifndef Py_LIMITED_API |
| 613 PyAPI_FUNC(int) _PyUnicode_Ready( | |
|
72529
2fc483009f6d
_PyUnicode_Ready() cannot be used on ready strings anymore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72528
diff
changeset
|
614 PyObject *unicode /* Unicode object */ |
| 72475 | 615 ); |
| 616 #endif | |
| 617 | |
|
72546
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
618 /* Get a copy of a Unicode string. */ |
|
73942
9c76bb654809
Make PyUnicode_Copy() private => _PyUnicode_Copy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73941
diff
changeset
|
619 #ifndef Py_LIMITED_API |
|
9c76bb654809
Make PyUnicode_Copy() private => _PyUnicode_Copy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73941
diff
changeset
|
620 PyAPI_FUNC(PyObject*) _PyUnicode_Copy( |
|
72546
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
621 PyObject *unicode |
|
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
622 ); |
|
73942
9c76bb654809
Make PyUnicode_Copy() private => _PyUnicode_Copy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73941
diff
changeset
|
623 #endif |
|
72546
c75b04996f54
Add PyUnicode_Copy() function, include it to the public API
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72529
diff
changeset
|
624 |
| 72475 | 625 /* Copy character from one unicode object into another, this function performs |
|
72492
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
626 character conversion when necessary and falls back to memcpy if possible. |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
627 |
|
72527
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
628 Fail if to is too small (smaller than how_many or smaller than |
|
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
629 len(from)-from_start), or if kind(from[from_start:from_start+how_many]) > |
|
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
630 kind(to), or if to has more than 1 reference. |
|
72492
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
631 |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
632 Return the number of written character, or return -1 and raise an exception |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
633 on error. |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
634 |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
635 Pseudo-code: |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
636 |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
637 how_many = min(how_many, len(from) - from_start) |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
638 to[to_start:to_start+how_many] = from[from_start:from_start+how_many] |
|
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
639 return how_many |
|
72527
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
640 |
|
a11970b158f2
Add a note in PyUnicode_CopyCharacters() doc: it doesn't write null character
Victor Stinner <vstinner@wyplay.com>
parents:
72506
diff
changeset
|
641 Note: The function doesn't write a terminating null character. |
|
72492
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
642 */ |
| 72475 | 643 #ifndef Py_LIMITED_API |
|
72492
e69b7f21823f
PyUnicode_CopyCharacters() checks for buffer and character overflow
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72491
diff
changeset
|
644 PyAPI_FUNC(Py_ssize_t) PyUnicode_CopyCharacters( |
| 72475 | 645 PyObject *to, |
| 646 Py_ssize_t to_start, | |
| 647 PyObject *from, | |
| 648 Py_ssize_t from_start, | |
| 649 Py_ssize_t how_many | |
| 650 ); | |
| 651 #endif | |
| 652 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
653 /* Create a Unicode Object from the Py_UNICODE buffer u of the given |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
654 size. |
|
17618
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
655 |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
656 u may be NULL which causes the contents to be undefined. It is the |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
657 user's responsibility to fill in the needed data afterwards. Note |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
658 that modifying the Unicode object contents after construction is |
|
b44ba7c01217
This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents:
13917
diff
changeset
|
659 only allowed if u was set to NULL. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
660 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
661 The buffer is copied into the new object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
662 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
663 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
664 PyAPI_FUNC(PyObject*) PyUnicode_FromUnicode( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
665 const Py_UNICODE *u, /* Unicode buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
666 Py_ssize_t size /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
667 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
668 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
669 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
670 /* Similar to PyUnicode_FromUnicode(), but u points to UTF-8 encoded bytes */ |
|
41766
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
671 PyAPI_FUNC(PyObject*) PyUnicode_FromStringAndSize( |
|
67152
f61196453c78
Issue #9738: Ooops, fix typos in my previous commit (r87506)
Victor Stinner <victor.stinner@haypocalc.com>
parents:
67150
diff
changeset
|
672 const char *u, /* UTF-8 encoded string */ |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
673 Py_ssize_t size /* size of buffer */ |
|
41766
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
674 ); |
|
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
675 |
|
41619
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
676 /* Similar to PyUnicode_FromUnicode(), but u points to null-terminated |
| 72475 | 677 UTF-8 encoded bytes. The size is determined with strlen(). */ |
|
41619
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
678 PyAPI_FUNC(PyObject*) PyUnicode_FromString( |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
679 const char *u /* UTF-8 encoded string */ |
|
41619
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
680 ); |
|
61acfb831633
Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents:
41587
diff
changeset
|
681 |
|
72685
61ca5262b6c4
Speedup str[a:b] and PyUnicode_FromKindAndData
Victor Stinner <vstinner@wyplay.com>
parents:
72632
diff
changeset
|
682 /* Create a new string from a buffer of Py_UCS1, Py_UCS2 or Py_UCS4 characters. |
|
61ca5262b6c4
Speedup str[a:b] and PyUnicode_FromKindAndData
Victor Stinner <vstinner@wyplay.com>
parents:
72632
diff
changeset
|
683 Scan the string to find the maximum character. */ |
| 72475 | 684 #ifndef Py_LIMITED_API |
| 685 PyAPI_FUNC(PyObject*) PyUnicode_FromKindAndData( | |
| 686 int kind, | |
| 687 const void *buffer, | |
| 688 Py_ssize_t size); | |
| 689 #endif | |
| 690 | |
| 691 PyAPI_FUNC(PyObject*) PyUnicode_Substring( | |
| 692 PyObject *str, | |
| 693 Py_ssize_t start, | |
| 694 Py_ssize_t end); | |
| 695 | |
|
72785
fe65c75c9f9d
Update C API docs for PEP 393.
Georg Brandl <georg@python.org>
parents:
72752
diff
changeset
|
696 /* Copy the string into a UCS4 buffer including the null character if copy_null |
| 72475 | 697 is set. Return NULL and raise an exception on error. Raise a ValueError if |
| 698 the buffer is smaller than the string. Return buffer on success. | |
| 699 | |
| 700 buflen is the length of the buffer in (Py_UCS4) characters. */ | |
| 701 PyAPI_FUNC(Py_UCS4*) PyUnicode_AsUCS4( | |
| 702 PyObject *unicode, | |
| 703 Py_UCS4* buffer, | |
| 704 Py_ssize_t buflen, | |
| 705 int copy_null); | |
| 706 | |
| 707 /* Copy the string into a UCS4 buffer. A new buffer is allocated using | |
| 708 * PyMem_Malloc; if this fails, NULL is returned with a memory error | |
| 709 exception set. */ | |
| 710 PyAPI_FUNC(Py_UCS4*) PyUnicode_AsUCS4Copy(PyObject *unicode); | |
| 711 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
712 /* Return a read-only pointer to the Unicode object's internal |
| 72475 | 713 Py_UNICODE buffer. |
| 714 If the wchar_t/Py_UNICODE representation is not yet available, this | |
| 715 function will calculate it. */ | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
716 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
717 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
718 PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicode( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
719 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
720 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
721 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
722 |
| 72475 | 723 /* Return a read-only pointer to the Unicode object's internal |
| 724 Py_UNICODE buffer and save the length at size. | |
| 725 If the wchar_t/Py_UNICODE representation is not yet available, this | |
| 726 function will calculate it. */ | |
| 727 | |
| 728 #ifndef Py_LIMITED_API | |
| 729 PyAPI_FUNC(Py_UNICODE *) PyUnicode_AsUnicodeAndSize( | |
| 730 PyObject *unicode, /* Unicode object */ | |
| 731 Py_ssize_t *size /* location where to save the length */ | |
| 732 ); | |
| 733 #endif | |
| 734 | |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
735 /* Get the length of the Unicode object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
736 |
| 72475 | 737 PyAPI_FUNC(Py_ssize_t) PyUnicode_GetLength( |
| 738 PyObject *unicode | |
| 739 ); | |
| 740 | |
|
72494
93cc591cffe4
Strip trailing spaces in unicodeobject.[ch]
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72492
diff
changeset
|
741 /* Get the number of Py_UNICODE units in the |
| 72475 | 742 string representation. */ |
| 743 | |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
744 PyAPI_FUNC(Py_ssize_t) PyUnicode_GetSize( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
745 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
746 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
747 |
| 72475 | 748 /* Read a character from the string. */ |
| 749 | |
| 750 PyAPI_FUNC(Py_UCS4) PyUnicode_ReadChar( | |
| 751 PyObject *unicode, | |
| 752 Py_ssize_t index | |
| 753 ); | |
| 754 | |
| 755 /* Write a character to the string. The string must have been created through | |
|
72575
99aa46107a22
PyUnicode_WriteChar() raises IndexError on invalid index
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72559
diff
changeset
|
756 PyUnicode_New, must not be shared, and must not have been hashed yet. |
|
99aa46107a22
PyUnicode_WriteChar() raises IndexError on invalid index
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72559
diff
changeset
|
757 |
|
99aa46107a22
PyUnicode_WriteChar() raises IndexError on invalid index
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72559
diff
changeset
|
758 Return 0 on success, -1 on error. */ |
| 72475 | 759 |
| 760 PyAPI_FUNC(int) PyUnicode_WriteChar( | |
| 761 PyObject *unicode, | |
| 762 Py_ssize_t index, | |
| 763 Py_UCS4 character | |
| 764 ); | |
| 765 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
766 #ifndef Py_LIMITED_API |
|
18298
982163c4e9d2
Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents:
18296
diff
changeset
|
767 /* Get the maximum ordinal for a Unicode character. */ |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
768 PyAPI_FUNC(Py_UNICODE) PyUnicode_GetMax(void); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
769 #endif |
|
18298
982163c4e9d2
Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents:
18296
diff
changeset
|
770 |
|
73946
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
771 /* Resize an Unicode object. The length is the number of characters, except |
|
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
772 if the kind of the string is PyUnicode_WCHAR_KIND: in this case, the length |
|
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
773 is the number of Py_UNICODE characters. |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
774 |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
775 *unicode is modified to point to the new (resized) object and 0 |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
776 returned on success. |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
777 |
|
73946
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
778 Try to resize the string in place (which is usually faster than allocating |
|
61a2cae5308c
Fix PyUnicode_Resize() for compact string: leave the string unchanged on error
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73942
diff
changeset
|
779 a new string and copy characters), or create a new string. |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
780 |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
781 Error handling is implemented as follows: an exception is set, -1 |
|
73947
3ad7d01acbf4
PyUnicode_Resize(): warn about canonical representation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73946
diff
changeset
|
782 is returned and *unicode left untouched. |
|
3ad7d01acbf4
PyUnicode_Resize(): warn about canonical representation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73946
diff
changeset
|
783 |
|
3ad7d01acbf4
PyUnicode_Resize(): warn about canonical representation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73946
diff
changeset
|
784 WARNING: The function doesn't check string content, the result may not be a |
|
3ad7d01acbf4
PyUnicode_Resize(): warn about canonical representation
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73946
diff
changeset
|
785 string in canonical representation. */ |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
786 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
787 PyAPI_FUNC(int) PyUnicode_Resize( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
788 PyObject **unicode, /* Pointer to the Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
789 Py_ssize_t length /* New length */ |
|
12326
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
790 ); |
|
9488e4fdafe4
Marc-Andre Lemburg: New exported API PyUnicode_Resize().
Guido van Rossum <guido@python.org>
parents:
12248
diff
changeset
|
791 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
792 /* Coerce obj to an Unicode object and return a reference with |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
793 *incremented* refcount. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
794 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
795 Coercion is done in the following way: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
796 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
797 1. bytes, bytearray and other char buffer compatible objects are decoded |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
798 under the assumptions that they contain data using the UTF-8 |
|
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
799 encoding. Decoding is done in "strict" mode. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
800 |
|
20554
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
801 2. All other objects (including Unicode objects) raise an |
|
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
802 exception. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
803 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
804 The API returns NULL in case of an error. The caller is responsible |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
805 for decref'ing the returned objects. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
806 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
807 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
808 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
809 PyAPI_FUNC(PyObject*) PyUnicode_FromEncodedObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
810 register PyObject *obj, /* Object */ |
|
13401
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
811 const char *encoding, /* encoding */ |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
812 const char *errors /* error handling */ |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
813 ); |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
814 |
|
20554
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
815 /* Coerce obj to an Unicode object and return a reference with |
|
13401
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
816 *incremented* refcount. |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
817 |
|
20554
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
818 Unicode objects are passed back as-is (subclasses are converted to |
|
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
819 true Unicode objects), all other objects are delegated to |
|
0336ee5b46d4
SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents:
19998
diff
changeset
|
820 PyUnicode_FromEncodedObject(obj, NULL, "strict") which results in |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
821 using UTF-8 encoding as basis for decoding the object. |
|
13401
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
822 |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
823 The API returns NULL in case of an error. The caller is responsible |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
824 for decref'ing the returned objects. |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
825 |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
826 */ |
|
8f1a2deeb9b0
Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents:
13393
diff
changeset
|
827 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
828 PyAPI_FUNC(PyObject*) PyUnicode_FromObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
829 register PyObject *obj /* Object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
830 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
831 |
|
64639
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
832 PyAPI_FUNC(PyObject *) PyUnicode_FromFormatV( |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
833 const char *format, /* ASCII-encoded string */ |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
834 va_list vargs |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
835 ); |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
836 PyAPI_FUNC(PyObject *) PyUnicode_FromFormat( |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
837 const char *format, /* ASCII-encoded string */ |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
838 ... |
|
1cfad55de5bd
Issue #9738: PyUnicode_FromFormat() and PyErr_Format() raise an error on
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64397
diff
changeset
|
839 ); |
|
41766
d6aa7edf217a
Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents:
41619
diff
changeset
|
840 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
841 #ifndef Py_LIMITED_API |
|
47393
0da27fdf0192
Refactor and clean up str.format() code (and helpers) in advance of optimizations.
Eric Smith <eric@trueblade.com>
parents:
46909
diff
changeset
|
842 /* Format the object based on the format_spec, as defined in PEP 3101 |
|
0da27fdf0192
Refactor and clean up str.format() code (and helpers) in advance of optimizations.
Eric Smith <eric@trueblade.com>
parents:
46909
diff
changeset
|
843 (Advanced String Formatting). */ |
|
0da27fdf0192
Refactor and clean up str.format() code (and helpers) in advance of optimizations.
Eric Smith <eric@trueblade.com>
parents:
46909
diff
changeset
|
844 PyAPI_FUNC(PyObject *) _PyUnicode_FormatAdvanced(PyObject *obj, |
| 72475 | 845 PyObject *format_spec, |
| 846 Py_ssize_t start, | |
| 847 Py_ssize_t end); | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
848 #endif |
|
47393
0da27fdf0192
Refactor and clean up str.format() code (and helpers) in advance of optimizations.
Eric Smith <eric@trueblade.com>
parents:
46909
diff
changeset
|
849 |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
850 PyAPI_FUNC(void) PyUnicode_InternInPlace(PyObject **); |
|
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
851 PyAPI_FUNC(void) PyUnicode_InternImmortal(PyObject **); |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
852 PyAPI_FUNC(PyObject *) PyUnicode_InternFromString( |
|
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
853 const char *u /* UTF-8 encoded string */ |
|
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
854 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
855 #ifndef Py_LIMITED_API |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
856 PyAPI_FUNC(void) _Py_ReleaseInternedUnicodeStrings(void); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
857 #endif |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
858 |
|
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
859 /* Use only if you know it's a string */ |
| 72475 | 860 #define PyUnicode_CHECK_INTERNED(op) \ |
| 861 (((PyASCIIObject *)(op))->state.interned) | |
|
41876
dcb43f40390b
Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents:
41769
diff
changeset
|
862 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
863 /* --- wchar_t support for platforms which support it --------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
864 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
865 #ifdef HAVE_WCHAR_H |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
866 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
867 /* Create a Unicode Object from the wchar_t buffer w of the given |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
868 size. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
869 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
870 The buffer is copied into the new object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
871 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
872 PyAPI_FUNC(PyObject*) PyUnicode_FromWideChar( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
873 register const wchar_t *w, /* wchar_t buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
874 Py_ssize_t size /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
875 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
876 |
|
34016
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
877 /* Copies the Unicode Object contents into the wchar_t buffer w. At |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
878 most size wchar_t characters are copied. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
879 |
|
34016
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
880 Note that the resulting wchar_t string may or may not be |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
881 0-terminated. It is the responsibility of the caller to make sure |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
882 that the wchar_t string is 0-terminated in case this is required by |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
883 the application. |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
884 |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
885 Returns the number of wchar_t characters copied (excluding a |
|
264e45baacd5
Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents:
33864
diff
changeset
|
886 possibly trailing 0-termination character) or -1 in case of an |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
887 error. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
888 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
889 PyAPI_FUNC(Py_ssize_t) PyUnicode_AsWideChar( |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
890 PyObject *unicode, /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
891 register wchar_t *w, /* wchar_t buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
892 Py_ssize_t size /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
893 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
894 |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
895 /* Convert the Unicode object to a wide character string. The output string |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
896 always ends with a nul character. If size is not NULL, write the number of |
|
72298
028423c89f8d
Fix PyUnicode_AsWideCharString() doc: size doesn't contain the null character
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68257
diff
changeset
|
897 wide characters (excluding the null character) into *size. |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
898 |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
899 Returns a buffer allocated by PyMem_Alloc() (use PyMem_Free() to free it) |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
900 on success. On error, returns NULL, *size is undefined and raises a |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
901 MemoryError. */ |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
902 |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
903 PyAPI_FUNC(wchar_t*) PyUnicode_AsWideCharString( |
|
65157
e3ae5b00ba15
PyUnicode_AsWideCharString() takes a PyObject*, not a PyUnicodeObject*
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65007
diff
changeset
|
904 PyObject *unicode, /* Unicode object */ |
|
65007
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
905 Py_ssize_t *size /* number of characters of the result */ |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
906 ); |
|
7c7d1d96e2c3
Issue #9979: Create function PyUnicode_AsWideCharString().
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64687
diff
changeset
|
907 |
|
72559
fc7d2c6db61b
_PyUnicode_AsKind() is *not* part of the stable ABI
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72554
diff
changeset
|
908 #ifndef Py_LIMITED_API |
| 72475 | 909 PyAPI_FUNC(void*) _PyUnicode_AsKind(PyObject *s, unsigned int kind); |
|
72559
fc7d2c6db61b
_PyUnicode_AsKind() is *not* part of the stable ABI
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72554
diff
changeset
|
910 #endif |
| 72475 | 911 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
912 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
913 |
|
24737
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
914 /* --- Unicode ordinals --------------------------------------------------- */ |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
915 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
916 /* Create a Unicode Object from the given Unicode code point ordinal. |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
917 |
|
24737
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
918 The ordinal must be in range(0x10000) on narrow Python builds |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
919 (UCS2), and range(0x110000) on wide builds (UCS4). A ValueError is |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
920 raised in case it is not. |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
921 |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
922 */ |
|
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
923 |
|
24753
27ae978bee58
Add name mangling for new PyUnicode_FromOrdinal() and fix declaration
Marc-André Lemburg <mal@egenix.com>
parents:
24752
diff
changeset
|
924 PyAPI_FUNC(PyObject*) PyUnicode_FromOrdinal(int ordinal); |
|
24737
eaa87632b1e8
Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents:
23561
diff
changeset
|
925 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
926 /* --- Free-list management ----------------------------------------------- */ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
927 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
928 /* Clear the free list used by the Unicode implementation. |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
929 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
930 This can be used to release memory used for objects on the free |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
931 list back to the Python memory allocator. |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
932 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
933 */ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
934 |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
935 PyAPI_FUNC(int) PyUnicode_ClearFreeList(void); |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
936 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
937 /* === Builtin Codecs ===================================================== |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
938 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
939 Many of these APIs take two arguments encoding and errors. These |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
940 parameters encoding and errors have the same semantics as the ones |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
941 of the builtin str() API. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
942 |
|
65113
c9879b3402e4
Merged revisions 82262,82269,82434,82480-82481,82484-82485,82487-82488,82594,82599,82615 via svnmerge from
Georg Brandl <georg@python.org>
parents:
61577
diff
changeset
|
943 Setting encoding to NULL causes the default encoding (UTF-8) to be used. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
944 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
945 Error handling is set by errors which may also be set to NULL |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
946 meaning to use the default handling defined for the codec. Default |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
947 error handling for all builtin codecs is "strict" (ValueErrors are |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
948 raised). |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
949 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
950 The codecs all use a similar interface. Only deviation from the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
951 generic ones are documented. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
952 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
953 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
954 |
|
12643
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
955 /* --- Manage the default encoding ---------------------------------------- */ |
|
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
956 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
957 /* Returns a pointer to the default encoding (UTF-8) of the |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
958 Unicode object unicode and the size of the encoded representation |
|
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
959 in bytes stored in *size. |
|
43761
658cd34a8c5d
Fixed a bug in PyUnicode_DecodeFSDefault. strcmp() returns 0 on success.
Christian Heimes <christian@cheimes.de>
parents:
43569
diff
changeset
|
960 |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
961 In case of an error, no *size is set. |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
962 |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
963 This function caches the UTF-8 encoded string in the unicodeobject |
|
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
964 and subsequent calls will return the same string. The memory is released |
| 72475 | 965 when the unicodeobject is deallocated. |
| 966 | |
| 967 _PyUnicode_AsStringAndSize is a #define for PyUnicode_AsUTF8AndSize to | |
| 968 support the previous internal function with the same behaviour. | |
| 969 | |
|
48660
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
970 *** This API is for interpreter INTERNAL USE ONLY and will likely |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
971 *** be removed or changed in the future. |
|
48660
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
972 |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
973 *** If you need to access the Unicode object as UTF-8 bytes string, |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
974 *** please use PyUnicode_AsUTF8String() instead. |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
975 */ |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
976 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
977 #ifndef Py_LIMITED_API |
| 72475 | 978 PyAPI_FUNC(char *) PyUnicode_AsUTF8AndSize( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
979 PyObject *unicode, |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
980 Py_ssize_t *size); |
| 72475 | 981 #define _PyUnicode_AsStringAndSize PyUnicode_AsUTF8AndSize |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
982 #endif |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
983 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
984 /* Returns a pointer to the default encoding (UTF-8) of the |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
985 Unicode object unicode. |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
986 |
| 72475 | 987 Like PyUnicode_AsUTF8AndSize(), this also caches the UTF-8 representation |
| 988 in the unicodeobject. | |
| 989 | |
| 990 _PyUnicode_AsString is a #define for PyUnicode_AsUTF8 to | |
| 991 support the previous internal function with the same behaviour. | |
| 992 | |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
993 Use of this API is DEPRECATED since no size information can be |
|
48660
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
994 extracted from the returned data. |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
995 |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
996 *** This API is for interpreter INTERNAL USE ONLY and will likely |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
997 *** be removed or changed for Python 3.1. |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
998 |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
999 *** If you need to access the Unicode object as UTF-8 bytes string, |
|
85b67fb5672b
Rename PyUnicode_AsString -> _PyUnicode_AsString and
Marc-André Lemburg <mal@egenix.com>
parents:
48358
diff
changeset
|
1000 *** please use PyUnicode_AsUTF8String() instead. |
|
42984
3a56e73ae919
Add PyUnicode_AsStringAndSize(), which is like PyUnicode_AsString() but
Guido van Rossum <guido@python.org>
parents:
42896
diff
changeset
|
1001 |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1002 */ |
|
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1003 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1004 #ifndef Py_LIMITED_API |
| 72475 | 1005 PyAPI_FUNC(char *) PyUnicode_AsUTF8(PyObject *unicode); |
| 1006 #define _PyUnicode_AsString PyUnicode_AsUTF8 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1007 #endif |
|
46598
0154df250ef6
Cleanup the Unicode header documentation and formatting a bit.
Marc-André Lemburg <mal@egenix.com>
parents:
45371
diff
changeset
|
1008 |
|
66206
52fe60216e89
Merged revisions 86478 via svnmerge from
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
65113
diff
changeset
|
1009 /* Returns "utf-8". */ |
|
12643
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
1010 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1011 PyAPI_FUNC(const char*) PyUnicode_GetDefaultEncoding(void); |
|
12643
06ff966f69d5
M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents:
12372
diff
changeset
|
1012 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1013 /* --- Generic Codecs ----------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1014 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1015 /* Create a Unicode object by decoding the encoded string s of the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1016 given size. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1017 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1018 PyAPI_FUNC(PyObject*) PyUnicode_Decode( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1019 const char *s, /* encoded string */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1020 Py_ssize_t size, /* size of buffer */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1021 const char *encoding, /* encoding */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1022 const char *errors /* error handling */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1023 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1024 |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1025 /* Decode a Unicode object unicode and return the result as Python |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1026 object. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1027 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1028 PyAPI_FUNC(PyObject*) PyUnicode_AsDecodedObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1029 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1030 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1031 const char *errors /* error handling */ |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1032 ); |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1033 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1034 /* Decode a Unicode object unicode and return the result as Unicode |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1035 object. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1036 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1037 PyAPI_FUNC(PyObject*) PyUnicode_AsDecodedUnicode( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1038 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1039 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1040 const char *errors /* error handling */ |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1041 ); |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1042 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1043 /* Encodes a Py_UNICODE buffer of the given size and returns a |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1044 Python string object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1045 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1046 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1047 PyAPI_FUNC(PyObject*) PyUnicode_Encode( |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1048 const Py_UNICODE *s, /* Unicode char buffer */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1049 Py_ssize_t size, /* number of Py_UNICODE chars to encode */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1050 const char *encoding, /* encoding */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1051 const char *errors /* error handling */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1052 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1053 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1054 |
|
32551
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1055 /* Encodes a Unicode object and returns the result as Python |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1056 object. */ |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1057 |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1058 PyAPI_FUNC(PyObject*) PyUnicode_AsEncodedObject( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1059 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1060 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1061 const char *errors /* error handling */ |
|
32551
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1062 ); |
|
f46d49e2e0f0
Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents:
32197
diff
changeset
|
1063 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1064 /* Encodes a Unicode object and returns the result as Python string |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1065 object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1066 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1067 PyAPI_FUNC(PyObject*) PyUnicode_AsEncodedString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1068 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1069 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1070 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1071 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1072 |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1073 /* Encodes a Unicode object and returns the result as Unicode |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1074 object. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1075 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1076 PyAPI_FUNC(PyObject*) PyUnicode_AsEncodedUnicode( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1077 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1078 const char *encoding, /* encoding */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1079 const char *errors /* error handling */ |
|
47496
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1080 ); |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1081 |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1082 /* Build an encoding map. */ |
|
2284fa89ab08
Move the codec decode type checks to bytes/bytearray.decode().
Marc-André Lemburg <mal@egenix.com>
parents:
47463
diff
changeset
|
1083 |
|
38673
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1084 PyAPI_FUNC(PyObject*) PyUnicode_BuildEncodingMap( |
|
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1085 PyObject* string /* 256 character map */ |
|
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1086 ); |
|
a19c20c5e5c7
Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents:
38669
diff
changeset
|
1087 |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1088 /* --- UTF-7 Codecs ------------------------------------------------------- */ |
|
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1089 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1090 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF7( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1091 const char *string, /* UTF-7 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1092 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1093 const char *errors /* error handling */ |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1094 ); |
|
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1095 |
|
43933
9c4bf18e5d30
Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
43761
diff
changeset
|
1096 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF7Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1097 const char *string, /* UTF-7 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1098 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1099 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1100 Py_ssize_t *consumed /* bytes consumed */ |
|
43933
9c4bf18e5d30
Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
43761
diff
changeset
|
1101 ); |
|
9c4bf18e5d30
Merged revisions 59056-59076 via svnmerge from
Christian Heimes <christian@cheimes.de>
parents:
43761
diff
changeset
|
1102 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1103 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1104 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF7( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1105 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1106 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1107 int base64SetO, /* Encode RFC2152 Set O characters in base64 */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1108 int base64WhiteSpace, /* Encode whitespace (sp, ht, nl, cr) in base64 */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1109 const char *errors /* error handling */ |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1110 ); |
|
73476
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1111 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF7( |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1112 PyObject *unicode, /* Unicode object */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1113 int base64SetO, /* Encode RFC2152 Set O characters in base64 */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1114 int base64WhiteSpace, /* Encode whitespace (sp, ht, nl, cr) in base64 */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1115 const char *errors /* error handling */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1116 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1117 #endif |
|
19998
8a0c97453df4
Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents:
19980
diff
changeset
|
1118 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1119 /* --- UTF-8 Codecs ------------------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1120 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1121 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1122 const char *string, /* UTF-8 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1123 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1124 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1125 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1126 |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1127 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF8Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1128 const char *string, /* UTF-8 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1129 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1130 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1131 Py_ssize_t *consumed /* bytes consumed */ |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1132 ); |
|
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1133 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1134 PyAPI_FUNC(PyObject*) PyUnicode_AsUTF8String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1135 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1136 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1137 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1138 #ifndef Py_LIMITED_API |
| 72475 | 1139 PyAPI_FUNC(PyObject*) _PyUnicode_AsUTF8String( |
| 1140 PyObject *unicode, | |
| 1141 const char *errors); | |
| 1142 | |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1143 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF8( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1144 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1145 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1146 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1147 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1148 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1149 |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1150 /* --- UTF-32 Codecs ------------------------------------------------------ */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1151 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1152 /* Decodes length bytes from a UTF-32 encoded buffer string and returns |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1153 the corresponding Unicode object. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1154 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1155 errors (if non-NULL) defines the error handling. It defaults |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1156 to "strict". |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1157 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1158 If byteorder is non-NULL, the decoder starts decoding using the |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1159 given byte order: |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1160 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1161 *byteorder == -1: little endian |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1162 *byteorder == 0: native order |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1163 *byteorder == 1: big endian |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1164 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1165 In native mode, the first four bytes of the stream are checked for a |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1166 BOM mark. If found, the BOM mark is analysed, the byte order |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1167 adjusted and the BOM skipped. In the other modes, no BOM mark |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1168 interpretation is done. After completion, *byteorder is set to the |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1169 current byte order at the end of input data. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1170 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1171 If byteorder is NULL, the codec starts in native order mode. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1172 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1173 */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1174 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1175 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF32( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1176 const char *string, /* UTF-32 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1177 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1178 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1179 int *byteorder /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1180 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1181 exit */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1182 ); |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1183 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1184 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF32Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1185 const char *string, /* UTF-32 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1186 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1187 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1188 int *byteorder, /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1189 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1190 exit */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1191 Py_ssize_t *consumed /* bytes consumed */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1192 ); |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1193 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1194 /* Returns a Python string using the UTF-32 encoding in native byte |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1195 order. The string always starts with a BOM mark. */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1196 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1197 PyAPI_FUNC(PyObject*) PyUnicode_AsUTF32String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1198 PyObject *unicode /* Unicode object */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1199 ); |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1200 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1201 /* Returns a Python string object holding the UTF-32 encoded value of |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1202 the Unicode data. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1203 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1204 If byteorder is not 0, output is written according to the following |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1205 byte order: |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1206 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1207 byteorder == -1: little endian |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1208 byteorder == 0: native byte order (writes a BOM mark) |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1209 byteorder == 1: big endian |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1210 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1211 If byteorder is 0, the output string will always start with the |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1212 Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1213 prepended. |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1214 |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1215 */ |
|
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1216 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1217 #ifndef Py_LIMITED_API |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1218 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF32( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1219 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1220 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1221 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1222 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1223 ); |
|
73476
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1224 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF32( |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1225 PyObject *object, /* Unicode object */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1226 const char *errors, /* error handling */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1227 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1228 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1229 #endif |
|
42626
f1a2d51f1cab
Apply SF patch #1775604: This adds three new codecs (utf-32, utf-32-le and
Walter Dörwald <walter@livinglogic.de>
parents:
42597
diff
changeset
|
1230 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1231 /* --- UTF-16 Codecs ------------------------------------------------------ */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1232 |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1233 /* Decodes length bytes from a UTF-16 encoded buffer string and returns |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1234 the corresponding Unicode object. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1235 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1236 errors (if non-NULL) defines the error handling. It defaults |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1237 to "strict". |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1238 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1239 If byteorder is non-NULL, the decoder starts decoding using the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1240 given byte order: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1241 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1242 *byteorder == -1: little endian |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1243 *byteorder == 0: native order |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1244 *byteorder == 1: big endian |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1245 |
|
17887
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1246 In native mode, the first two bytes of the stream are checked for a |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1247 BOM mark. If found, the BOM mark is analysed, the byte order |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1248 adjusted and the BOM skipped. In the other modes, no BOM mark |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1249 interpretation is done. After completion, *byteorder is set to the |
|
fd5f83188778
This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents:
17618
diff
changeset
|
1250 current byte order at the end of input data. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1251 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1252 If byteorder is NULL, the codec starts in native order mode. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1253 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1254 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1255 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1256 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF16( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1257 const char *string, /* UTF-16 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1258 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1259 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1260 int *byteorder /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1261 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1262 exit */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1263 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1264 |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1265 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUTF16Stateful( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1266 const char *string, /* UTF-16 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1267 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1268 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1269 int *byteorder, /* pointer to byteorder to use |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1270 0=native;-1=LE,1=BE; updated on |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1271 exit */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1272 Py_ssize_t *consumed /* bytes consumed */ |
|
33428
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1273 ); |
|
57478a13fbf6
SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents:
32951
diff
changeset
|
1274 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1275 /* Returns a Python string using the UTF-16 encoding in native byte |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1276 order. The string always starts with a BOM mark. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1277 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1278 PyAPI_FUNC(PyObject*) PyUnicode_AsUTF16String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1279 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1280 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1281 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1282 /* Returns a Python string object holding the UTF-16 encoded value of |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1283 the Unicode data. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1284 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1285 If byteorder is not 0, output is written according to the following |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1286 byte order: |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1287 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1288 byteorder == -1: little endian |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1289 byteorder == 0: native byte order (writes a BOM mark) |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1290 byteorder == 1: big endian |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1291 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1292 If byteorder is 0, the output string will always start with the |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1293 Unicode BOM mark (U+FEFF). In the other two modes, no BOM mark is |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1294 prepended. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1295 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1296 Note that Py_UNICODE data is being interpreted as UTF-16 reduced to |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1297 UCS-2. This trick makes it possible to add full UTF-16 capabilities |
|
13620
5b727ab37092
Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents:
13401
diff
changeset
|
1298 at a later point without compromising the APIs. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1299 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1300 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1301 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1302 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1303 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUTF16( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1304 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1305 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1306 const char *errors, /* error handling */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1307 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1308 ); |
|
73476
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1309 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeUTF16( |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1310 PyObject* unicode, /* Unicode object */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1311 const char *errors, /* error handling */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1312 int byteorder /* byteorder to use 0=BOM+native;-1=LE,1=BE */ |
|
b3e532c1485e
Port encoders from Py_UNICODE API to unicode object API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73457
diff
changeset
|
1313 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1314 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1315 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1316 /* --- Unicode-Escape Codecs ---------------------------------------------- */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1317 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1318 PyAPI_FUNC(PyObject*) PyUnicode_DecodeUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1319 const char *string, /* Unicode-Escape encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1320 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1321 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1322 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1323 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1324 PyAPI_FUNC(PyObject*) PyUnicode_AsUnicodeEscapeString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1325 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1326 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1327 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1328 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1329 PyAPI_FUNC(PyObject*) PyUnicode_EncodeUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1330 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1331 Py_ssize_t length /* Number of Py_UNICODE chars to encode */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1332 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1333 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1334 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1335 /* --- Raw-Unicode-Escape Codecs ------------------------------------------ */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1336 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1337 PyAPI_FUNC(PyObject*) PyUnicode_DecodeRawUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1338 const char *string, /* Raw-Unicode-Escape encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1339 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1340 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1341 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1342 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1343 PyAPI_FUNC(PyObject*) PyUnicode_AsRawUnicodeEscapeString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1344 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1345 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1346 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1347 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1348 PyAPI_FUNC(PyObject*) PyUnicode_EncodeRawUnicodeEscape( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1349 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1350 Py_ssize_t length /* Number of Py_UNICODE chars to encode */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1351 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1352 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1353 |
| 35485 | 1354 /* --- Unicode Internal Codec --------------------------------------------- |
| 1355 | |
| 1356 Only for internal use in _codecsmodule.c */ | |
| 1357 | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1358 #ifndef Py_LIMITED_API |
| 35485 | 1359 PyObject *_PyUnicode_DecodeUnicodeInternal( |
| 1360 const char *string, | |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1361 Py_ssize_t length, |
| 35485 | 1362 const char *errors |
| 1363 ); | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1364 #endif |
| 35485 | 1365 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1366 /* --- Latin-1 Codecs ----------------------------------------------------- |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1367 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1368 Note: Latin-1 corresponds to the first 256 Unicode ordinals. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1369 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1370 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1371 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1372 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLatin1( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1373 const char *string, /* Latin-1 encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1374 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1375 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1376 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1377 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1378 PyAPI_FUNC(PyObject*) PyUnicode_AsLatin1String( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1379 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1380 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1381 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1382 #ifndef Py_LIMITED_API |
| 72475 | 1383 PyAPI_FUNC(PyObject*) _PyUnicode_AsLatin1String( |
| 1384 PyObject* unicode, | |
| 1385 const char* errors); | |
| 1386 | |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1387 PyAPI_FUNC(PyObject*) PyUnicode_EncodeLatin1( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1388 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1389 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1390 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1391 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1392 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1393 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1394 /* --- ASCII Codecs ------------------------------------------------------- |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1395 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1396 Only 7-bit ASCII data is excepted. All other codes generate errors. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1397 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1398 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1399 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1400 PyAPI_FUNC(PyObject*) PyUnicode_DecodeASCII( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1401 const char *string, /* ASCII encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1402 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1403 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1404 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1405 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1406 PyAPI_FUNC(PyObject*) PyUnicode_AsASCIIString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1407 PyObject *unicode /* Unicode object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1408 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1409 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1410 #ifndef Py_LIMITED_API |
| 72475 | 1411 PyAPI_FUNC(PyObject*) _PyUnicode_AsASCIIString( |
| 1412 PyObject* unicode, | |
| 1413 const char* errors); | |
| 1414 | |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1415 PyAPI_FUNC(PyObject*) PyUnicode_EncodeASCII( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1416 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1417 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1418 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1419 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1420 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1421 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1422 /* --- Character Map Codecs ----------------------------------------------- |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1423 |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1424 This codec uses mappings to encode and decode characters. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1425 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1426 Decoding mappings must map single string characters to single |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1427 Unicode characters, integers (which are then interpreted as Unicode |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1428 ordinals) or None (meaning "undefined mapping" and causing an |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1429 error). |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1430 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1431 Encoding mappings must map single Unicode characters to single |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1432 string characters, integers (which are then interpreted as Latin-1 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1433 ordinals) or None (meaning "undefined mapping" and causing an |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1434 error). |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1435 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1436 If a character lookup fails with a LookupError, the character is |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1437 copied as-is meaning that its ordinal value will be interpreted as |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1438 Unicode or Latin-1 ordinal resp. Because of this mappings only need |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1439 to contain those mappings which map characters to different code |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1440 points. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1441 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1442 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1443 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1444 PyAPI_FUNC(PyObject*) PyUnicode_DecodeCharmap( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1445 const char *string, /* Encoded string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1446 Py_ssize_t length, /* size of string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1447 PyObject *mapping, /* character mapping |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1448 (char ordinal -> unicode ordinal) */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1449 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1450 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1451 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1452 PyAPI_FUNC(PyObject*) PyUnicode_AsCharmapString( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1453 PyObject *unicode, /* Unicode object */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1454 PyObject *mapping /* character mapping |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1455 (unicode ordinal -> char ordinal) */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1456 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1457 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1458 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1459 PyAPI_FUNC(PyObject*) PyUnicode_EncodeCharmap( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1460 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1461 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1462 PyObject *mapping, /* character mapping |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1463 (unicode ordinal -> char ordinal) */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1464 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1465 ); |
|
73302
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1466 PyAPI_FUNC(PyObject*) _PyUnicode_EncodeCharmap( |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1467 PyObject *unicode, /* Unicode object */ |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1468 PyObject *mapping, /* character mapping |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1469 (unicode ordinal -> char ordinal) */ |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1470 const char *errors /* error handling */ |
|
295fdfd4f422
Port UCS1 and charmap codecs to new API.
Martin v. Löwis <martin@v.loewis.de>
parents:
73236
diff
changeset
|
1471 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1472 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1473 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1474 /* Translate a Py_UNICODE buffer of the given length by applying a |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1475 character mapping table to it and return the resulting Unicode |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1476 object. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1477 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1478 The mapping table must map Unicode ordinal integers to Unicode |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1479 ordinal integers or None (causing deletion of the character). |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1480 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1481 Mapping tables may be dictionaries or sequences. Unmapped character |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1482 ordinals (ones which cause a LookupError) are left untouched and |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1483 are copied as-is. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1484 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1485 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1486 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1487 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1488 PyAPI_FUNC(PyObject *) PyUnicode_TranslateCharmap( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1489 const Py_UNICODE *data, /* Unicode char buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1490 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1491 PyObject *table, /* Translate table */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1492 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1493 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1494 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1495 |
|
71193
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
1496 #ifdef HAVE_MBCS |
| 12074 | 1497 |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1498 /* --- MBCS codecs for Windows -------------------------------------------- */ |
| 12074 | 1499 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1500 PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCS( |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1501 const char *string, /* MBCS encoded string */ |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1502 Py_ssize_t length, /* size of string */ |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1503 const char *errors /* error handling */ |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1504 ); |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1505 |
|
39442
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1506 PyAPI_FUNC(PyObject*) PyUnicode_DecodeMBCSStateful( |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1507 const char *string, /* MBCS encoded string */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1508 Py_ssize_t length, /* size of string */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1509 const char *errors, /* error handling */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1510 Py_ssize_t *consumed /* bytes consumed */ |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1511 ); |
|
654c380cf8b9
Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents:
38673
diff
changeset
|
1512 |
|
72976
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1513 PyAPI_FUNC(PyObject*) PyUnicode_DecodeCodePageStateful( |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1514 int code_page, /* code page number */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1515 const char *string, /* encoded string */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1516 Py_ssize_t length, /* size of string */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1517 const char *errors, /* error handling */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1518 Py_ssize_t *consumed /* bytes consumed */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1519 ); |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1520 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1521 PyAPI_FUNC(PyObject*) PyUnicode_AsMBCSString( |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1522 PyObject *unicode /* Unicode object */ |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1523 ); |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1524 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1525 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1526 PyAPI_FUNC(PyObject*) PyUnicode_EncodeMBCS( |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1527 const Py_UNICODE *data, /* Unicode char buffer */ |
|
72976
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1528 Py_ssize_t length, /* number of Py_UNICODE chars to encode */ |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1529 const char *errors /* error handling */ |
|
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1530 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1531 #endif |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1532 |
|
72976
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1533 PyAPI_FUNC(PyObject*) PyUnicode_EncodeCodePage( |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1534 int code_page, /* code page number */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1535 PyObject *unicode, /* Unicode object */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1536 const char *errors /* error handling */ |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1537 ); |
|
af0800b986b7
Issue #12281: Rewrite the MBCS codec to handle correctly replace and ignore
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72925
diff
changeset
|
1538 |
|
71193
13e6d3cb2ecd
Issue #9642: Uniformize the tests on the availability of the mbcs codec
Victor Stinner <victor.stinner@haypocalc.com>
parents:
68258
diff
changeset
|
1539 #endif /* HAVE_MBCS */ |
|
12066
86cffe50fb17
Prototypes added for MBCS codecs. (Win32 only.)
Guido van Rossum <guido@python.org>
parents:
12025
diff
changeset
|
1540 |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1541 /* --- Decimal Encoder ---------------------------------------------------- */ |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1542 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1543 /* Takes a Unicode string holding a decimal value and writes it into |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1544 an output buffer using standard ASCII digit codes. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1545 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1546 The output buffer has to provide at least length+1 bytes of storage |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1547 area. The output string is 0-terminated. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1548 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1549 The encoder converts whitespace to ' ', decimal characters to their |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1550 corresponding ASCII digit and all other Latin-1 characters except |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1551 \0 as-is. Characters outside this range (Unicode ordinals 1-256) |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1552 are treated as errors. This includes embedded NULL bytes. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1553 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1554 Error handling is defined by the errors argument: |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1555 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1556 NULL or "strict": raise a ValueError |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1557 "ignore": ignore the wrong characters (these are not copied to the |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1558 output buffer) |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1559 "replace": replaces illegal characters with '?' |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1560 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1561 Returns 0 on success, -1 on failure. |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1562 |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1563 */ |
|
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1564 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1565 #ifndef Py_LIMITED_API |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1566 PyAPI_FUNC(int) PyUnicode_EncodeDecimal( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1567 Py_UNICODE *s, /* Unicode buffer */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1568 Py_ssize_t length, /* Number of Py_UNICODE chars to encode */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1569 char *output, /* Output buffer; must have size >= length */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1570 const char *errors /* error handling */ |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1571 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1572 #endif |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1573 |
|
66673
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1574 /* Transforms code points that have decimal digit property to the |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1575 corresponding ASCII digit code points. |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1576 |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1577 Returns a new Unicode string on success, NULL on failure. |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1578 */ |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1579 |
|
66751
902db4722594
Take PyUnicode_TransformDecimalToASCII out of the limited API.
Georg Brandl <georg@python.org>
parents:
66673
diff
changeset
|
1580 #ifndef Py_LIMITED_API |
|
66673
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1581 PyAPI_FUNC(PyObject*) PyUnicode_TransformDecimalToASCII( |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1582 Py_UNICODE *s, /* Unicode buffer */ |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1583 Py_ssize_t length /* Number of Py_UNICODE chars to transform */ |
|
7b6af56ee74d
Issue #10557: Fixed error messages from float() and other numeric
Alexander Belopolsky <alexander.belopolsky@gmail.com>
parents:
66664
diff
changeset
|
1584 ); |
|
66751
902db4722594
Take PyUnicode_TransformDecimalToASCII out of the limited API.
Georg Brandl <georg@python.org>
parents:
66673
diff
changeset
|
1585 #endif |
|
12248
70154e2e9c89
Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents:
12074
diff
changeset
|
1586 |
|
73591
abe62c9d659d
Fix misused of "PyUnicodeObject" structure name in unicodeobject.h
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73476
diff
changeset
|
1587 /* Similar to PyUnicode_TransformDecimalToASCII(), but takes a PyObject |
| 72475 | 1588 as argument instead of a raw buffer and length. This function additionally |
| 1589 transforms spaces to ASCII because this is what the callers in longobject, | |
| 1590 floatobject, and complexobject did anyways. */ | |
| 1591 | |
| 1592 #ifndef Py_LIMITED_API | |
| 1593 PyAPI_FUNC(PyObject*) _PyUnicode_TransformDecimalAndSpaceToASCII( | |
| 1594 PyObject *unicode /* Unicode object */ | |
| 1595 ); | |
| 1596 #endif | |
| 1597 | |
|
74002
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1598 /* --- Locale encoding --------------------------------------------------- */ |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1599 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1600 /* Decode a string from the current locale encoding. The decoder is strict if |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1601 *surrogateescape* is equal to zero, otherwise it uses the 'surrogateescape' |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1602 error handler (PEP 383) to escape undecodable bytes. If a byte sequence can |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1603 be decoded as a surrogate character and *surrogateescape* is not equal to |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1604 zero, the byte sequence is escaped using the 'surrogateescape' error handler |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1605 instead of being decoded. *str* must end with a null character but cannot |
|
74006
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1606 contain embedded null characters. */ |
|
74002
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1607 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1608 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocaleAndSize( |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1609 const char *str, |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1610 Py_ssize_t len, |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1611 int surrogateescape); |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1612 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1613 /* Similar to PyUnicode_DecodeLocaleAndSize(), but compute the string |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1614 length using strlen(). */ |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1615 |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1616 PyAPI_FUNC(PyObject*) PyUnicode_DecodeLocale( |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1617 const char *str, |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1618 int surrogateescape); |
|
279b0aee0cfb
Add PyUnicode_DecodeLocaleAndSize() and PyUnicode_DecodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73947
diff
changeset
|
1619 |
|
74006
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1620 /* Encode a Unicode object to the current locale encoding. The encoder is |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1621 strict is *surrogateescape* is equal to zero, otherwise the |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1622 "surrogateescape" error handler is used. Return a bytes object. The string |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1623 cannot contain embedded null characters.. */ |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1624 |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1625 PyAPI_FUNC(PyObject*) PyUnicode_EncodeLocale( |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1626 PyObject *unicode, |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1627 int surrogateescape |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1628 ); |
|
88198b93ff2f
Issue #13560: Add PyUnicode_EncodeLocale()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
74002
diff
changeset
|
1629 |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1630 /* --- File system encoding ---------------------------------------------- */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1631 |
|
63963
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1632 /* ParseTuple converter: encode str objects to bytes using |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1633 PyUnicode_EncodeFSDefault(); bytes objects are output as-is. */ |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1634 |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1635 PyAPI_FUNC(int) PyUnicode_FSConverter(PyObject*, void*); |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1636 |
|
63963
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1637 /* ParseTuple converter: decode bytes objects to unicode using |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1638 PyUnicode_DecodeFSDefaultAndSize(); str objects are output as-is. */ |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1639 |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1640 PyAPI_FUNC(int) PyUnicode_FSDecoder(PyObject*, void*); |
|
50c0dbaeaa04
Issue #9542: Create PyUnicode_FSDecoder() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63912
diff
changeset
|
1641 |
|
61577
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1642 /* Decode a null-terminated string using Py_FileSystemDefaultEncoding |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1643 and the "surrogateescape" error handler. |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1644 |
|
65348
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1645 If Py_FileSystemDefaultEncoding is not set, fall back to the locale |
|
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1646 encoding. |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1647 |
| 61601 | 1648 Use PyUnicode_DecodeFSDefaultAndSize() if the string length is known. |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1649 */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1650 |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1651 PyAPI_FUNC(PyObject*) PyUnicode_DecodeFSDefault( |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1652 const char *s /* encoded string */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1653 ); |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1654 |
|
61577
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1655 /* Decode a string using Py_FileSystemDefaultEncoding |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1656 and the "surrogateescape" error handler. |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1657 |
|
65348
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1658 If Py_FileSystemDefaultEncoding is not set, fall back to the locale |
|
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1659 encoding. |
|
61577
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1660 */ |
|
2b0af69aef97
Merged revisions 81168 via svnmerge from
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61467
diff
changeset
|
1661 |
|
53767
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1662 PyAPI_FUNC(PyObject*) PyUnicode_DecodeFSDefaultAndSize( |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1663 const char *s, /* encoded string */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1664 Py_ssize_t size /* size */ |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1665 ); |
|
ac209f132b17
Issue #5915: Implement PEP 383, Non-decodable Bytes in
Martin v. Löwis <martin@v.loewis.de>
parents:
53742
diff
changeset
|
1666 |
|
61597
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1667 /* Encode a Unicode object to Py_FileSystemDefaultEncoding with the |
| 61601 | 1668 "surrogateescape" error handler, and return bytes. |
|
61597
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1669 |
|
65348
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1670 If Py_FileSystemDefaultEncoding is not set, fall back to the locale |
|
453fd8182c2f
Use locale encoding if Py_FileSystemDefaultEncoding is not set
Victor Stinner <victor.stinner@haypocalc.com>
parents:
65157
diff
changeset
|
1671 encoding. |
|
61597
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1672 */ |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1673 |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1674 PyAPI_FUNC(PyObject*) PyUnicode_EncodeFSDefault( |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1675 PyObject *unicode |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1676 ); |
|
b5ba7ea4246b
Issue #8715: Create PyUnicode_EncodeFSDefault() function: Encode a Unicode
Victor Stinner <victor.stinner@haypocalc.com>
parents:
61576
diff
changeset
|
1677 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1678 /* --- Methods & Slots ---------------------------------------------------- |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1679 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1680 These are capable of handling Unicode objects and strings on input |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1681 (we refer to them as strings in the descriptions) and return |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
1682 Unicode objects or integers as appropriate. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1683 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1684 /* Concat two strings giving a new Unicode string. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1685 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1686 PyAPI_FUNC(PyObject*) PyUnicode_Concat( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1687 PyObject *left, /* Left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1688 PyObject *right /* Right string */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1689 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1690 |
|
41769
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1691 /* Concat two strings and put the result in *pleft |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1692 (sets *pleft to NULL on error) */ |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1693 |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1694 PyAPI_FUNC(void) PyUnicode_Append( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1695 PyObject **pleft, /* Pointer to left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1696 PyObject *right /* Right string */ |
|
41769
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1697 ); |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1698 |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1699 /* Concat two strings, put the result in *pleft and drop the right object |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1700 (sets *pleft to NULL on error) */ |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1701 |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1702 PyAPI_FUNC(void) PyUnicode_AppendAndDel( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1703 PyObject **pleft, /* Pointer to left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1704 PyObject *right /* Right string */ |
|
41769
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1705 ); |
|
b29d71d43c80
Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents:
41768
diff
changeset
|
1706 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1707 /* Split a string giving a list of Unicode strings. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1708 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1709 If sep is NULL, splitting will be done at all whitespace |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1710 substrings. Otherwise, splits occur at the given separator. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1711 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1712 At most maxsplit splits will be done. If negative, no limit is set. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1713 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1714 Separators are not included in the resulting list. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1715 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1716 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1717 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1718 PyAPI_FUNC(PyObject*) PyUnicode_Split( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1719 PyObject *s, /* String to split */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1720 PyObject *sep, /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1721 Py_ssize_t maxsplit /* Maxsplit count */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1722 ); |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1723 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1724 /* Dito, but split at line breaks. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1725 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1726 CRLF is considered to be one line break. Line breaks are not |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1727 included in the resulting list. */ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1728 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1729 PyAPI_FUNC(PyObject*) PyUnicode_Splitlines( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1730 PyObject *s, /* String to split */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1731 int keepends /* If true, line end markers are included */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1732 ); |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1733 |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1734 /* Partition a string using a given separator. */ |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1735 |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1736 PyAPI_FUNC(PyObject*) PyUnicode_Partition( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1737 PyObject *s, /* String to partition */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1738 PyObject *sep /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1739 ); |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1740 |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1741 /* Partition a string using a given separator, searching from the end of the |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1742 string. */ |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1743 |
|
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1744 PyAPI_FUNC(PyObject*) PyUnicode_RPartition( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1745 PyObject *s, /* String to partition */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1746 PyObject *sep /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1747 ); |
|
38453
29ae5e0fc348
Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents:
36501
diff
changeset
|
1748 |
|
31199
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1749 /* Split a string giving a list of Unicode strings. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1750 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1751 If sep is NULL, splitting will be done at all whitespace |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1752 substrings. Otherwise, splits occur at the given separator. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1753 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1754 At most maxsplit splits will be done. But unlike PyUnicode_Split |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1755 PyUnicode_RSplit splits from the end of the string. If negative, |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1756 no limit is set. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1757 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1758 Separators are not included in the resulting list. |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1759 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1760 */ |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1761 |
|
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1762 PyAPI_FUNC(PyObject*) PyUnicode_RSplit( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1763 PyObject *s, /* String to split */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1764 PyObject *sep, /* String separator */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1765 Py_ssize_t maxsplit /* Maxsplit count */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1766 ); |
|
31199
504943c89193
Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents:
24753
diff
changeset
|
1767 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1768 /* Translate a string by applying a character mapping table to it and |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1769 return the resulting Unicode object. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1770 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1771 The mapping table must map Unicode ordinal integers to Unicode |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1772 ordinal integers or None (causing deletion of the character). |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1773 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1774 Mapping tables may be dictionaries or sequences. Unmapped character |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1775 ordinals (ones which cause a LookupError) are left untouched and |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1776 are copied as-is. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1777 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1778 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1779 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1780 PyAPI_FUNC(PyObject *) PyUnicode_Translate( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1781 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1782 PyObject *table, /* Translate table */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1783 const char *errors /* error handling */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1784 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1785 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1786 /* Join a sequence of strings using the given separator and return |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1787 the resulting Unicode string. */ |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1788 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1789 PyAPI_FUNC(PyObject*) PyUnicode_Join( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1790 PyObject *separator, /* Separator string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1791 PyObject *seq /* Sequence object */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1792 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1793 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1794 /* Return 1 if substr matches str[start:end] at the given tail end, 0 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1795 otherwise. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1796 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1797 PyAPI_FUNC(Py_ssize_t) PyUnicode_Tailmatch( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1798 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1799 PyObject *substr, /* Prefix or Suffix string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1800 Py_ssize_t start, /* Start index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1801 Py_ssize_t end, /* Stop index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1802 int direction /* Tail end: -1 prefix, +1 suffix */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1803 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1804 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1805 /* Return the first position of substr in str[start:end] using the |
|
23561
c5e8f89fd358
Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents:
23225
diff
changeset
|
1806 given search direction or -1 if not found. -2 is returned in case |
|
c5e8f89fd358
Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents:
23225
diff
changeset
|
1807 an error occurred and an exception is set. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1808 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1809 PyAPI_FUNC(Py_ssize_t) PyUnicode_Find( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1810 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1811 PyObject *substr, /* Substring to find */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1812 Py_ssize_t start, /* Start index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1813 Py_ssize_t end, /* Stop index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1814 int direction /* Find direction: +1 forward, -1 backward */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1815 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1816 |
| 72475 | 1817 /* Like PyUnicode_Find, but search for single character only. */ |
| 1818 PyAPI_FUNC(Py_ssize_t) PyUnicode_FindChar( | |
| 1819 PyObject *str, | |
| 1820 Py_UCS4 ch, | |
| 1821 Py_ssize_t start, | |
| 1822 Py_ssize_t end, | |
| 1823 int direction | |
| 1824 ); | |
| 1825 | |
|
12025
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
1826 /* Count the number of occurrences of substr in str[start:end]. */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1827 |
|
36501
7fdc639bc5b4
Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents:
35808
diff
changeset
|
1828 PyAPI_FUNC(Py_ssize_t) PyUnicode_Count( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1829 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1830 PyObject *substr, /* Substring to count */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1831 Py_ssize_t start, /* Start index */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1832 Py_ssize_t end /* Stop index */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1833 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1834 |
|
12025
7994d9db7f4d
On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents:
12002
diff
changeset
|
1835 /* Replace at most maxcount occurrences of substr in str with replstr |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1836 and return the resulting Unicode object. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1837 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1838 PyAPI_FUNC(PyObject *) PyUnicode_Replace( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1839 PyObject *str, /* String */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1840 PyObject *substr, /* Substring to find */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1841 PyObject *replstr, /* Substring to replace */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1842 Py_ssize_t maxcount /* Max. number of replacements to apply; |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1843 -1 = all */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1844 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1845 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1846 /* Compare two strings and return -1, 0, 1 for less than, equal, |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1847 greater than resp. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1848 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1849 PyAPI_FUNC(int) PyUnicode_Compare( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1850 PyObject *left, /* Left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1851 PyObject *right /* Right string */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1852 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1853 |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1854 PyAPI_FUNC(int) PyUnicode_CompareWithASCIIString( |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1855 PyObject *left, |
|
67150
4bb7f7b151fe
Issue #9738: document encodings of unicode functions
Victor Stinner <victor.stinner@haypocalc.com>
parents:
66751
diff
changeset
|
1856 const char *right /* ASCII-encoded string */ |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1857 ); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
1858 |
|
39610
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1859 /* Rich compare two strings and return one of the following: |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1860 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1861 - NULL in case an exception was raised |
|
72689
aa79b4e62c38
Fix a few typos in the unicode header.
Georg Brandl <georg@python.org>
parents:
72688
diff
changeset
|
1862 - Py_True or Py_False for successfully comparisons |
|
39610
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1863 - Py_NotImplemented in case the type combination is unknown |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1864 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1865 Note that Py_EQ and Py_NE comparisons can cause a UnicodeWarning in |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1866 case the conversion of the arguments to Unicode fails with a |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1867 UnicodeDecodeError. |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1868 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1869 Possible values for op: |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1870 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1871 Py_GT, Py_GE, Py_EQ, Py_NE, Py_LT, Py_LE |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1872 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1873 */ |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1874 |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1875 PyAPI_FUNC(PyObject *) PyUnicode_RichCompare( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1876 PyObject *left, /* Left string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1877 PyObject *right, /* Right string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1878 int op /* Operation: Py_EQ, Py_NE, Py_GT, etc. */ |
|
39610
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1879 ); |
|
3674f20ac781
Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents:
39442
diff
changeset
|
1880 |
|
13620
5b727ab37092
Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents:
13401
diff
changeset
|
1881 /* Apply a argument tuple or dictionary to a format string and return |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1882 the resulting Unicode string. */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1883 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1884 PyAPI_FUNC(PyObject *) PyUnicode_Format( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1885 PyObject *format, /* Format string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1886 PyObject *args /* Argument tuple or dictionary */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1887 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1888 |
|
12002
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1889 /* Checks whether element is contained in container and return 1/0 |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1890 accordingly. |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1891 |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1892 element has to coerce to an one element Unicode string. -1 is |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1893 returned in case of an error. */ |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1894 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1895 PyAPI_FUNC(int) PyUnicode_Contains( |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1896 PyObject *container, /* Container string */ |
|
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1897 PyObject *element /* Element string */ |
|
12002
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1898 ); |
|
d50fa3670b66
Marc-Andre Lemburg: add declaration for PyUnicode_Contains().
Guido van Rossum <guido@python.org>
parents:
11962
diff
changeset
|
1899 |
|
42597
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
1900 /* Checks whether argument is a valid identifier. */ |
|
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
1901 |
|
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
1902 PyAPI_FUNC(int) PyUnicode_IsIdentifier(PyObject *s); |
|
20477cbfd996
Implement PEP 3131. Add isidentifier to str.
Martin v. Löwis <martin@v.loewis.de>
parents:
42595
diff
changeset
|
1903 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1904 #ifndef Py_LIMITED_API |
|
23225
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
1905 /* Externally visible for str.strip(unicode) */ |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1906 PyAPI_FUNC(PyObject *) _PyUnicode_XStrip( |
|
73185
f61db5671cc8
Replace PyUnicodeObject* by PyObject* where it was irrevelant
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72980
diff
changeset
|
1907 PyObject *self, |
|
23225
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
1908 int striptype, |
|
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
1909 PyObject *sepobj |
|
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
1910 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1911 #endif |
|
23225
e639e34ef901
Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents:
20554
diff
changeset
|
1912 |
|
46909
ef4ec968fdb6
Merged revisions 63078 via svnmerge from
Eric Smith <eric@trueblade.com>
parents:
46598
diff
changeset
|
1913 /* Using the current locale, insert the thousands grouping |
|
ef4ec968fdb6
Merged revisions 63078 via svnmerge from
Eric Smith <eric@trueblade.com>
parents:
46598
diff
changeset
|
1914 into the string pointed to by buffer. For the argument descriptions, |
|
ef4ec968fdb6
Merged revisions 63078 via svnmerge from
Eric Smith <eric@trueblade.com>
parents:
46598
diff
changeset
|
1915 see Objects/stringlib/localeutil.h */ |
|
ef4ec968fdb6
Merged revisions 63078 via svnmerge from
Eric Smith <eric@trueblade.com>
parents:
46598
diff
changeset
|
1916 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1917 #ifndef Py_LIMITED_API |
|
53187
b653eab13b73
The other half of Issue #1580: use short float repr where possible.
Eric Smith <eric@trueblade.com>
parents:
52773
diff
changeset
|
1918 PyAPI_FUNC(Py_ssize_t) _PyUnicode_InsertThousandsGroupingLocale(Py_UNICODE *buffer, |
|
b653eab13b73
The other half of Issue #1580: use short float repr where possible.
Eric Smith <eric@trueblade.com>
parents:
52773
diff
changeset
|
1919 Py_ssize_t n_buffer, |
|
b653eab13b73
The other half of Issue #1580: use short float repr where possible.
Eric Smith <eric@trueblade.com>
parents:
52773
diff
changeset
|
1920 Py_UNICODE *digits, |
|
b653eab13b73
The other half of Issue #1580: use short float repr where possible.
Eric Smith <eric@trueblade.com>
parents:
52773
diff
changeset
|
1921 Py_ssize_t n_digits, |
|
b653eab13b73
The other half of Issue #1580: use short float repr where possible.
Eric Smith <eric@trueblade.com>
parents:
52773
diff
changeset
|
1922 Py_ssize_t min_width); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1923 #endif |
|
46909
ef4ec968fdb6
Merged revisions 63078 via svnmerge from
Eric Smith <eric@trueblade.com>
parents:
46598
diff
changeset
|
1924 |
|
52773
28096ed73153
Added ',' thousands grouping to int.__format__. See PEP 378.
Eric Smith <eric@trueblade.com>
parents:
50660
diff
changeset
|
1925 /* Using explicit passed-in values, insert the thousands grouping |
|
28096ed73153
Added ',' thousands grouping to int.__format__. See PEP 378.
Eric Smith <eric@trueblade.com>
parents:
50660
diff
changeset
|
1926 into the string pointed to by buffer. For the argument descriptions, |
|
28096ed73153
Added ',' thousands grouping to int.__format__. See PEP 378.
Eric Smith <eric@trueblade.com>
parents:
50660
diff
changeset
|
1927 see Objects/stringlib/localeutil.h */ |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1928 #ifndef Py_LIMITED_API |
| 72475 | 1929 PyAPI_FUNC(Py_ssize_t) _PyUnicode_InsertThousandsGrouping( |
|
72705
05ed6e5f2cf4
Add asciilib: similar to ucs1, ucs2 and ucs4 library, but specialized to ASCII
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72699
diff
changeset
|
1930 PyObject *unicode, |
| 72475 | 1931 int kind, |
| 1932 void *buffer, | |
| 1933 Py_ssize_t n_buffer, | |
| 1934 void *digits, | |
| 1935 Py_ssize_t n_digits, | |
| 1936 Py_ssize_t min_width, | |
| 1937 const char *grouping, | |
| 1938 const char *thousands_sep); | |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1939 #endif |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1940 /* === Characters Type APIs =============================================== */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1941 |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1942 /* Helper array used by Py_UNICODE_ISSPACE(). */ |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1943 |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
1944 #ifndef Py_LIMITED_API |
|
50660
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1945 PyAPI_DATA(const unsigned char) _Py_ascii_whitespace[]; |
|
9d093ef89c56
Merged revisions 68440 via svnmerge from
Benjamin Peterson <benjamin@python.org>
parents:
50550
diff
changeset
|
1946 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1947 /* These should not be used directly. Use the Py_UNICODE_IS* and |
|
61467
f197407bdba2
Recorded merge of revisions 81032 via svnmerge from
Antoine Pitrou <solipsis@pitrou.net>
parents:
53767
diff
changeset
|
1948 Py_UNICODE_TO* macros instead. |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1949 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1950 These APIs are implemented in Objects/unicodectype.c. |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1951 |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1952 */ |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1953 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1954 PyAPI_FUNC(int) _PyUnicode_IsLowercase( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1955 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1956 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1957 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1958 PyAPI_FUNC(int) _PyUnicode_IsUppercase( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1959 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1960 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1961 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1962 PyAPI_FUNC(int) _PyUnicode_IsTitlecase( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1963 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1964 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1965 |
|
42595
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
1966 PyAPI_FUNC(int) _PyUnicode_IsXidStart( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1967 Py_UCS4 ch /* Unicode character */ |
|
42595
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
1968 ); |
|
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
1969 |
|
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
1970 PyAPI_FUNC(int) _PyUnicode_IsXidContinue( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1971 Py_UCS4 ch /* Unicode character */ |
|
42595
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
1972 ); |
|
f6af0b2598ca
Add XID_Start and XID_Continue properties to unicodectype.
Martin v. Löwis <martin@v.loewis.de>
parents:
42490
diff
changeset
|
1973 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1974 PyAPI_FUNC(int) _PyUnicode_IsWhitespace( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1975 const Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1976 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1977 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1978 PyAPI_FUNC(int) _PyUnicode_IsLinebreak( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1979 const Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1980 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1981 |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1982 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToLowercase( |
|
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1983 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1984 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1985 |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1986 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToUppercase( |
|
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1987 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1988 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1989 |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1990 PyAPI_FUNC(Py_UCS4) _PyUnicode_ToTitlecase( |
|
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1991 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1992 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1993 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1994 PyAPI_FUNC(int) _PyUnicode_ToDecimalDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1995 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1996 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
1997 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
1998 PyAPI_FUNC(int) _PyUnicode_ToDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
1999 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2000 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2001 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2002 PyAPI_FUNC(double) _PyUnicode_ToNumeric( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2003 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2004 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2005 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2006 PyAPI_FUNC(int) _PyUnicode_IsDecimalDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2007 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2008 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2009 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2010 PyAPI_FUNC(int) _PyUnicode_IsDigit( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2011 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2012 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2013 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2014 PyAPI_FUNC(int) _PyUnicode_IsNumeric( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2015 Py_UCS4 ch /* Unicode character */ |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2016 ); |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2017 |
|
47611
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
2018 PyAPI_FUNC(int) _PyUnicode_IsPrintable( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2019 Py_UCS4 ch /* Unicode character */ |
|
47611
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
2020 ); |
|
da47e7e135ae
#2630: Implement PEP 3138.
Georg Brandl <georg@python.org>
parents:
47496
diff
changeset
|
2021 |
|
24752
c1eb668525e1
Excise DL_EXPORT from Include.
Mark Hammond <mhammond@skippinet.com.au>
parents:
24737
diff
changeset
|
2022 PyAPI_FUNC(int) _PyUnicode_IsAlpha( |
|
64146
8f97c64b2333
#5127: Even on narrow unicode builds, the C functions that access the Unicode
Amaury Forgeot d'Arc <amauryfa@gmail.com>
parents:
64081
diff
changeset
|
2023 Py_UCS4 ch /* Unicode character */ |
|
13352
d88535ccaee5
Modified the ISALPHA and ISALNUM macros to use the new lookup APIs
Marc-André Lemburg <mal@egenix.com>
parents:
13320
diff
changeset
|
2024 ); |
|
d88535ccaee5
Modified the ISALPHA and ISALNUM macros to use the new lookup APIs
Marc-André Lemburg <mal@egenix.com>
parents:
13320
diff
changeset
|
2025 |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2026 PyAPI_FUNC(size_t) Py_UNICODE_strlen( |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2027 const Py_UNICODE *u |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2028 ); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2029 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2030 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strcpy( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2031 Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2032 const Py_UNICODE *s2); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2033 |
|
64372
a0d8de04c3a8
Create Py_UNICODE_strcat() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64349
diff
changeset
|
2034 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strcat( |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2035 Py_UNICODE *s1, const Py_UNICODE *s2); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2036 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2037 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strncpy( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2038 Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2039 const Py_UNICODE *s2, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2040 size_t n); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2041 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2042 PyAPI_FUNC(int) Py_UNICODE_strcmp( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2043 const Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2044 const Py_UNICODE *s2 |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2045 ); |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2046 |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2047 PyAPI_FUNC(int) Py_UNICODE_strncmp( |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2048 const Py_UNICODE *s1, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2049 const Py_UNICODE *s2, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2050 size_t n |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2051 ); |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2052 |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2053 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strchr( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2054 const Py_UNICODE *s, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2055 Py_UNICODE c |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2056 ); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2057 |
|
63912
a3853bfa2518
Issue #9425: create Py_UNICODE_strrchr() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
62530
diff
changeset
|
2058 PyAPI_FUNC(Py_UNICODE*) Py_UNICODE_strrchr( |
|
64081
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2059 const Py_UNICODE *s, |
|
b089ed576b13
Issue #9425: Create Py_UNICODE_strncmp() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
63963
diff
changeset
|
2060 Py_UNICODE c |
|
42013
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2061 ); |
|
cd125fe83051
Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents:
41876
diff
changeset
|
2062 |
|
64373
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2063 /* Create a copy of a unicode string ending with a nul character. Return NULL |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2064 and raise a MemoryError exception on memory allocation failure, otherwise |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2065 return a new allocated buffer (use PyMem_Free() to free the buffer). */ |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2066 |
|
64397
98f337a6957c
Rename PyUnicode_strdup() to PyUnicode_AsUnicodeCopy()
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64373
diff
changeset
|
2067 PyAPI_FUNC(Py_UNICODE*) PyUnicode_AsUnicodeCopy( |
|
64373
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2068 PyObject *unicode |
|
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2069 ); |
|
66664
62b61abd02b8
Merge branches/pep-0384.
Martin v. Löwis <martin@v.loewis.de>
parents:
66203
diff
changeset
|
2070 #endif /* Py_LIMITED_API */ |
|
64373
1f04b0076fd8
Create PyUnicode_strdup() function
Victor Stinner <victor.stinner@haypocalc.com>
parents:
64372
diff
changeset
|
2071 |
|
72714
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2072 #if defined(Py_DEBUG) && !defined(Py_LIMITED_API) |
|
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2073 PyAPI_FUNC(int) _PyUnicode_CheckConsistency( |
|
73337
783b4b9dfa5e
Replace PyUnicodeObject type by PyObject
Victor Stinner <victor.stinner@haypocalc.com>
parents:
73302
diff
changeset
|
2074 PyObject *op, |
|
72714
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2075 int check_content); |
|
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2076 #endif |
|
6f03716079a9
Don't check for the maximum character when copying from unicodeobject.c
Victor Stinner <victor.stinner@haypocalc.com>
parents:
72705
diff
changeset
|
2077 |
|
72835
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2078 /********************* String Literals ****************************************/ |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2079 /* This structure helps managing static strings. The basic usage goes like this: |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2080 Instead of doing |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2081 |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2082 r = PyObject_CallMethod(o, "foo", "args", ...); |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2083 |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2084 do |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2085 |
|
72925
7109f31300fb
Rename _Py_identifier to _Py_IDENTIFIER.
Martin v. Löwis <martin@v.loewis.de>
parents:
72904
diff
changeset
|
2086 _Py_IDENTIFIER(foo); |
|
72835
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2087 ... |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2088 r = _PyObject_CallMethodId(o, &PyId_foo, "args", ...); |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2089 |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2090 PyId_foo is a static variable, either on block level or file level. On first |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2091 usage, the string "foo" is interned, and the structures are linked. On interpreter |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2092 shutdown, all strings are released (through _PyUnicode_ClearStaticStrings). |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2093 |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2094 Alternatively, _Py_static_string allows to choose the variable name. |
|
73457
6ae8c68eb0c1
Make _PyUnicode_FromId return borrowed references.
Martin v. Löwis <martin@v.loewis.de>
parents:
73356
diff
changeset
|
2095 _PyUnicode_FromId returns a borrowed reference to the interned string. |
|
72835
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2096 _PyObject_{Get,Set,Has}AttrId are __getattr__ versions using _Py_Identifier*. |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2097 */ |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2098 typedef struct _Py_Identifier { |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2099 struct _Py_Identifier *next; |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2100 const char* string; |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2101 PyObject *object; |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2102 } _Py_Identifier; |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2103 |
|
72836
57ed1e24f8f3
Drop extra semicolon.
Martin v. Löwis <martin@v.loewis.de>
parents:
72835
diff
changeset
|
2104 #define _Py_static_string(varname, value) static _Py_Identifier varname = { 0, value, 0 } |
|
72925
7109f31300fb
Rename _Py_identifier to _Py_IDENTIFIER.
Martin v. Löwis <martin@v.loewis.de>
parents:
72904
diff
changeset
|
2105 #define _Py_IDENTIFIER(varname) _Py_static_string(PyId_##varname, #varname) |
|
72835
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2106 |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2107 /* Return an interned Unicode object for an Identifier; may fail if there is no memory.*/ |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2108 PyAPI_FUNC(PyObject*) _PyUnicode_FromId(_Py_Identifier*); |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2109 /* Clear all static strings. */ |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2110 PyAPI_FUNC(void) _PyUnicode_ClearStaticStrings(void); |
|
5e33219492bc
Add API for static strings, primarily good for identifiers.
Martin v. Löwis <martin@v.loewis.de>
parents:
72796
diff
changeset
|
2111 |
|
11962
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2112 #ifdef __cplusplus |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2113 } |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2114 #endif |
|
be98e055e39d
Unicode implementation by Marc-Andre Lemburg based on original code by Fredrik Lundh.
Guido van Rossum <guido@python.org>
parents:
diff
changeset
|
2115 #endif /* !Py_UNICODEOBJECT_H */ |