Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodeobject.c @ 42457:080f2aa18015

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Change PyUnicode_FromString[AndSize] to expect UTF-8.
author Martin v. Löwis <martin@v.loewis.de>
date Sun, 05 Aug 2007 20:26:11 +0000
parents 504084c04ac0
children 3fc3cedc49ca
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1 /*
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 Unicode implementation based on original code by Fredrik Lundh,
12646
17d1a3f1f085 Minimal change so I can add the rest of MAL's checkin message:
Fred Drake <fdrake@acm.org>
parents: 12645
diff changeset
4 modified by Marc-Andre Lemburg <mal@lemburg.com> according to the
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 Unicode Integration Proposal (see file Misc/unicode.txt).
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7 Major speed upgrades to the method implementations at the Reykjavik
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8 NeedForSpeed sprint, by Fredrik Lundh and Andrew Dalke.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
9
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13761
diff changeset
10 Copyright (c) Corporation for National Research Initiatives.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
11
16029
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
12 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
13 The original string type implementation is:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
14
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
15 Copyright (c) 1999 by Secret Labs AB
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
16 Copyright (c) 1999 by Fredrik Lundh
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
17
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
18 By obtaining, using, and/or copying this software and/or its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
19 associated documentation, you agree that you have read, understood,
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
20 and will comply with the following terms and conditions:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
21
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
22 Permission to use, copy, modify, and distribute this software and its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
23 associated documentation for any purpose and without fee is hereby
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
24 granted, provided that the above copyright notice appears in all
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
25 copies, and that both that copyright notice and this permission notice
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
26 appear in supporting documentation, and that the name of Secret Labs
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
27 AB or the author not be used in advertising or publicity pertaining to
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
28 distribution of the software without specific, written prior
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
29 permission.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
30
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
31 SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
32 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
33 FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
34 ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
35 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
36 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
37 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
38 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
39
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
40 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
41
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
42 #define PY_SSIZE_T_CLEAN
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
43 #include "Python.h"
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
44
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
45 #include "unicodeobject.h"
13209
b11e0a898de0 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13196
diff changeset
46 #include "ucnhash.h"
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
47
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
48 #ifdef MS_WINDOWS
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
49 #include <windows.h>
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
50 #endif
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
51
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 /* Limit for the Unicode object free list */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
53
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 #define MAX_UNICODE_FREELIST_SIZE 1024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
55
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
56 /* Limit for the Unicode object free list stay alive optimization.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
57
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 The implementation will keep allocated Unicode memory intact for
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
59 all objects on the free list having a size less than this
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
60 limit. This reduces malloc() overhead for small Unicode objects.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
61
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
62 At worst this will result in MAX_UNICODE_FREELIST_SIZE *
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
63 (sizeof(PyUnicodeObject) + KEEPALIVE_SIZE_LIMIT +
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
64 malloc()-overhead) bytes of unused garbage.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
65
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
66 Setting the limit to 0 effectively turns the feature off.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
67
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
68 Note: This is an experimental feature ! If you get core dumps when
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
69 using Unicode objects, turn this feature off.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
70
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
72
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
73 #define KEEPALIVE_SIZE_LIMIT 9
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
74
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 /* Endianness switches; defaults to little endian */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
76
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 #ifdef WORDS_BIGENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 # define BYTEORDER_IS_BIG_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 # define BYTEORDER_IS_LITTLE_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
81 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
82
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
83 /* --- Globals ------------------------------------------------------------
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
84
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
85 The globals are initialized by the _PyUnicode_Init() API and should
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
86 not be used before calling that API.
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
87
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
88 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
89
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
90
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
91 #ifdef __cplusplus
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
92 extern "C" {
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
93 #endif
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
94
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
95 /* This dictionary holds all interned unicode strings. Note that references
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
96 to strings in this dictionary are *not* counted in the string's ob_refcnt.
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
97 When the interned string reaches a refcnt of 0 the string deallocation
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
98 function will delete the reference from this dictionary.
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
99
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
100 Another way to look at this is that to say that the actual reference
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
101 count of a string is: s->ob_refcnt + (s->ob_sstate?2:0)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
102 */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
103 static PyObject *interned;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
104
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
105 /* Free list for Unicode objects */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
106 static PyUnicodeObject *unicode_freelist;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
107 static int unicode_freelist_size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
108
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
109 /* The empty Unicode object is shared to improve performance. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
110 static PyUnicodeObject *unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
111
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
112 /* Single character Unicode strings in the Latin-1 range are being
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
113 shared as well. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
114 static PyUnicodeObject *unicode_latin1[256];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
115
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
116 /* Default encoding to use and assume when NULL is passed as encoding
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
117 parameter; it is fixed to "utf-8". Always use the
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
118 PyUnicode_GetDefaultEncoding() API to access this global. */
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
119 static const char unicode_default_encoding[] = "utf-8";
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
120
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
121 Py_UNICODE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
122 PyUnicode_GetMax(void)
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
123 {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
124 #ifdef Py_UNICODE_WIDE
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
125 return 0x10FFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
126 #else
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
127 /* This is actually an illegal character, so it should
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
128 not be passed to unichr. */
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
129 return 0xFFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
130 #endif
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
131 }
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
132
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
133 /* --- Bloom Filters ----------------------------------------------------- */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
134
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
135 /* stuff to implement simple "bloom filters" for Unicode characters.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
136 to keep things simple, we use a single bitmask, using the least 5
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
137 bits from each unicode characters as the bit index. */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
138
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
139 /* the linebreak mask is set up by Unicode_Init below */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
140
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
141 #define BLOOM_MASK unsigned long
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
142
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
143 static BLOOM_MASK bloom_linebreak;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
144
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
145 #define BLOOM(mask, ch) ((mask & (1 << ((ch) & 0x1F))))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
146
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
147 #define BLOOM_LINEBREAK(ch)\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
148 (BLOOM(bloom_linebreak, (ch)) && Py_UNICODE_ISLINEBREAK((ch)))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
149
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
150 Py_LOCAL_INLINE(BLOOM_MASK) make_bloom_mask(Py_UNICODE* ptr, Py_ssize_t len)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
151 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
152 /* calculate simple bloom-style bitmask for a given unicode string */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
153
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
154 long mask;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
155 Py_ssize_t i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
156
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
157 mask = 0;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
158 for (i = 0; i < len; i++)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
159 mask |= (1 << (ptr[i] & 0x1F));
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
160
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
161 return mask;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
162 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
163
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
164 Py_LOCAL_INLINE(int) unicode_member(Py_UNICODE chr, Py_UNICODE* set, Py_ssize_t setlen)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
165 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
166 Py_ssize_t i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
167
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
168 for (i = 0; i < setlen; i++)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
169 if (set[i] == chr)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
170 return 1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
171
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
172 return 0;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
173 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
174
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
175 #define BLOOM_MEMBER(mask, chr, set, setlen)\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
176 BLOOM(mask, chr) && unicode_member(chr, set, setlen)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
177
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
178 /* --- Unicode Object ----------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
179
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
180 static
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
181 int unicode_resize(register PyUnicodeObject *unicode,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
182 Py_ssize_t length)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
183 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
184 void *oldstr;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
185
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
186 /* Shortcut if there's nothing much to do. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
187 if (unicode->length == length)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
188 goto reset;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
189
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
190 /* Resizing shared object (unicode_empty or single character
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
191 objects) in-place is not allowed. Use PyUnicode_Resize()
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
192 instead ! */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
193
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
194 if (unicode == unicode_empty ||
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
195 (unicode->length == 1 &&
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
196 unicode->str[0] < 256U &&
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
197 unicode_latin1[unicode->str[0]] == unicode)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
198 PyErr_SetString(PyExc_SystemError,
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
199 "can't resize shared unicode objects");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
200 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
202
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
203 /* We allocate one more byte to make sure the string is Ux0000 terminated.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
204 The overallocation is also used by fastsearch, which assumes that it's
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
205 safe to look at str[length] (without making any assumptions about what
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
206 it contains). */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
207
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
208 oldstr = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
209 PyMem_RESIZE(unicode->str, Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
210 if (!unicode->str) {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
211 unicode->str = (Py_UNICODE *)oldstr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
212 PyErr_NoMemory();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
213 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
214 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
215 unicode->str[length] = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
216 unicode->length = length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
217
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
218 reset:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
219 /* Reset the object caches */
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
220 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
221 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
222 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
223 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
224 unicode->hash = -1;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
225
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
226 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
227 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
228
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
229 /* We allocate one more byte to make sure the string is
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
230 Ux0000 terminated -- XXX is this needed ?
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
231
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
232 XXX This allocator could further be enhanced by assuring that the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
233 free list never reduces its size below 1.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
234
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
235 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
236
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
237 static
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
238 PyUnicodeObject *_PyUnicode_New(Py_ssize_t length)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
239 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
240 register PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
241
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
242 /* Optimization for empty strings */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
243 if (length == 0 && unicode_empty != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
244 Py_INCREF(unicode_empty);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
245 return unicode_empty;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
246 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
247
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
248 /* Unicode freelist & memory allocation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
249 if (unicode_freelist) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
250 unicode = unicode_freelist;
12939
88004f25dc25 Vladimir MARANGOZOV <Vladimir.Marangozov@inrialpes.fr>:
Marc-André Lemburg <mal@egenix.com>
parents: 12924
diff changeset
251 unicode_freelist = *(PyUnicodeObject **)unicode;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
252 unicode_freelist_size--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
253 if (unicode->str) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
254 /* Keep-Alive optimization: we only upsize the buffer,
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
255 never downsize it. */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
256 if ((unicode->length < length) &&
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
257 unicode_resize(unicode, length) < 0) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
258 PyMem_DEL(unicode->str);
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
259 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
260 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
261 }
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
262 else {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
263 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
264 }
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
265 PyObject_INIT(unicode, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
266 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
267 else {
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
268 unicode = PyObject_New(PyUnicodeObject, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
269 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
270 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
271 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
272 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
273
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
274 if (!unicode->str) {
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
275 PyErr_NoMemory();
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
276 goto onError;
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
277 }
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
278 /* Initialize the first element to guard against cases where
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
279 * the caller fails before initializing str -- unicode_resize()
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
280 * reads str[0], and the Keep-Alive optimization can keep memory
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
281 * allocated for str alive across a call to unicode_dealloc(unicode).
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
282 * We don't want unicode_resize to read uninitialized memory in
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
283 * that case.
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
284 */
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
285 unicode->str[0] = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
286 unicode->str[length] = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
287 unicode->length = length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
288 unicode->hash = -1;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
289 unicode->state = 0;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
290 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
291 return unicode;
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
292
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
293 onError:
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
294 _Py_ForgetReference((PyObject *)unicode);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
295 PyObject_Del(unicode);
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
296 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
297 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
298
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
299 static
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
300 void unicode_dealloc(register PyUnicodeObject *unicode)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
301 {
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
302 switch (PyUnicode_CHECK_INTERNED(unicode)) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
303 case SSTATE_NOT_INTERNED:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
304 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
305
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
306 case SSTATE_INTERNED_MORTAL:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
307 /* revive dead object temporarily for DelItem */
42335
c8b850b5a066 Fix merge breakage.
Martin v. Löwis <martin@v.loewis.de>
parents: 42334
diff changeset
308 Py_Refcnt(unicode) = 3;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
309 if (PyDict_DelItem(interned, (PyObject *)unicode) != 0)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
310 Py_FatalError(
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
311 "deletion of interned unicode string failed");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
312 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
313
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
314 case SSTATE_INTERNED_IMMORTAL:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
315 Py_FatalError("Immortal interned unicode string died.");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
316
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
317 default:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
318 Py_FatalError("Inconsistent interned unicode string state.");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
319 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
320
21262
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
321 if (PyUnicode_CheckExact(unicode) &&
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
322 unicode_freelist_size < MAX_UNICODE_FREELIST_SIZE) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
323 /* Keep-Alive optimization */
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
324 if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
325 PyMem_DEL(unicode->str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
326 unicode->str = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
327 unicode->length = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
328 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
329 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
330 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
331 unicode->defenc = NULL;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
332 }
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
333 /* Add to free list */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
334 *(PyUnicodeObject **)unicode = unicode_freelist;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
335 unicode_freelist = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
336 unicode_freelist_size++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
337 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
338 else {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
339 PyMem_DEL(unicode->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
340 Py_XDECREF(unicode->defenc);
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
341 Py_Type(unicode)->tp_free((PyObject *)unicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
342 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
343 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
344
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
345 int PyUnicode_Resize(PyObject **unicode, Py_ssize_t length)
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
346 {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
347 register PyUnicodeObject *v;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
348
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
349 /* Argument checks */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
350 if (unicode == NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
351 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
352 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
353 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
354 v = (PyUnicodeObject *)*unicode;
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
355 if (v == NULL || !PyUnicode_Check(v) || Py_Refcnt(v) != 1 || length < 0) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
356 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
357 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
358 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
359
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
360 /* Resizing unicode_empty and single character objects is not
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
361 possible since these are being shared. We simply return a fresh
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
362 copy with the same Unicode content. */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
363 if (v->length != length &&
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
364 (v == unicode_empty || v->length == 1)) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
365 PyUnicodeObject *w = _PyUnicode_New(length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
366 if (w == NULL)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
367 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
368 Py_UNICODE_COPY(w->str, v->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
369 length < v->length ? length : v->length);
28242
fe1d571c3f80 Sf patch #700047: unicode object leaks refcount on resizing
Raymond Hettinger <python@rcn.com>
parents: 27826
diff changeset
370 Py_DECREF(*unicode);
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
371 *unicode = (PyObject *)w;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
372 return 0;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
373 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
374
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
375 /* Note that we don't have to modify *unicode for unshared Unicode
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
376 objects, since we can modify them in-place. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
377 return unicode_resize(v, length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
378 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
379
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
380 /* Internal API for use in unicodeobject.c only ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
381 #define _PyUnicode_Resize(unicodevar, length) \
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
382 PyUnicode_Resize(((PyObject **)(unicodevar)), length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
383
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
384 PyObject *PyUnicode_FromUnicode(const Py_UNICODE *u,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
385 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
386 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
387 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
388
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
389 /* If the Unicode data is known at construction time, we can apply
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
390 some optimizations which share commonly used objects. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
391 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
392
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
393 /* Optimization for empty strings */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
394 if (size == 0 && unicode_empty != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
395 Py_INCREF(unicode_empty);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
396 return (PyObject *)unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
397 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
398
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
399 /* Single character Unicode objects in the Latin-1 range are
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
400 shared when using this constructor */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
401 if (size == 1 && *u < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
402 unicode = unicode_latin1[*u];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
403 if (!unicode) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
404 unicode = _PyUnicode_New(1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
405 if (!unicode)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
406 return NULL;
18051
e897550b4748 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode.
Marc-André Lemburg <mal@egenix.com>
parents: 17995
diff changeset
407 unicode->str[0] = *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
408 unicode_latin1[*u] = unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
409 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
410 Py_INCREF(unicode);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
411 return (PyObject *)unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
412 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
413 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
414
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
415 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
416 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
417 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
418
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
419 /* Copy the Unicode data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
420 if (u != NULL)
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
421 Py_UNICODE_COPY(unicode->str, u, size);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
422
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
423 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
424 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
425
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
426 PyObject *PyUnicode_FromStringAndSize(const char *u, Py_ssize_t size)
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
427 {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
428 PyUnicodeObject *unicode;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
429 /* If the Unicode data is known at construction time, we can apply
42457
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
430 some optimizations which share commonly used objects.
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
431 Also, this means the input must be UTF-8, so fall back to the
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
432 UTF-8 decoder at the end. */
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
433 if (u != NULL) {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
434
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
435 /* Optimization for empty strings */
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
436 if (size == 0 && unicode_empty != NULL) {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
437 Py_INCREF(unicode_empty);
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
438 return (PyObject *)unicode_empty;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
439 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
440
42457
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
441 /* Single characters are shared when using this constructor.
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
442 Restrict to ASCII, since the input must be UTF-8. */
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
443 if (size == 1 && Py_CHARMASK(*u) < 128) {
42318
4999a0947343 Fix a bug in PyUnicode_FromStringAndSize() with signed characters.
Guido van Rossum <guido@python.org>
parents: 42309
diff changeset
444 unicode = unicode_latin1[Py_CHARMASK(*u)];
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
445 if (!unicode) {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
446 unicode = _PyUnicode_New(1);
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
447 if (!unicode)
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
448 return NULL;
42318
4999a0947343 Fix a bug in PyUnicode_FromStringAndSize() with signed characters.
Guido van Rossum <guido@python.org>
parents: 42309
diff changeset
449 unicode->str[0] = Py_CHARMASK(*u);
4999a0947343 Fix a bug in PyUnicode_FromStringAndSize() with signed characters.
Guido van Rossum <guido@python.org>
parents: 42309
diff changeset
450 unicode_latin1[Py_CHARMASK(*u)] = unicode;
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
451 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
452 Py_INCREF(unicode);
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
453 return (PyObject *)unicode;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
454 }
42457
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
455
080f2aa18015 Change PyUnicode_FromString[AndSize] to expect UTF-8.
Martin v. Löwis <martin@v.loewis.de>
parents: 42408
diff changeset
456 return PyUnicode_DecodeUTF8(u, size, NULL);
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
457 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
458
41765
141e2aeffbaa Revert last checkin: _PyUnicode_New() allocates space
Walter Dörwald <walter@livinglogic.de>
parents: 41764
diff changeset
459 unicode = _PyUnicode_New(size);
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
460 if (!unicode)
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
461 return NULL;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
462
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
463 return (PyObject *)unicode;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
464 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
465
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
466 PyObject *PyUnicode_FromString(const char *u)
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
467 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
468 size_t size = strlen(u);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
469 if (size > PY_SSIZE_T_MAX) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
470 PyErr_SetString(PyExc_OverflowError, "input too long");
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
471 return NULL;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
472 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
473
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
474 return PyUnicode_FromStringAndSize(u, size);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
475 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
476
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
477 #ifdef HAVE_WCHAR_H
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
478
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
479 PyObject *PyUnicode_FromWideChar(register const wchar_t *w,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
480 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
481 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
482 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
483
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
484 if (w == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
485 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
486 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
487 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
488
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
489 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
490 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
491 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
492
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
493 /* Copy the wchar_t data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
494 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
495 memcpy(unicode->str, w, size * sizeof(wchar_t));
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
496 #else
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
497 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
498 register Py_UNICODE *u;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
499 register Py_ssize_t i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
500 u = PyUnicode_AS_UNICODE(unicode);
33766
1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
Marc-André Lemburg <mal@egenix.com>
parents: 33499
diff changeset
501 for (i = size; i > 0; i--)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
502 *u++ = *w++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
503 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
504 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
505
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
506 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
507 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
508
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
509 static void
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
510 makefmt(char *fmt, int longflag, int size_tflag, int zeropad, int width, int precision, char c)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
511 {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
512 *fmt++ = '%';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
513 if (width) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
514 if (zeropad)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
515 *fmt++ = '0';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
516 fmt += sprintf(fmt, "%d", width);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
517 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
518 if (precision)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
519 fmt += sprintf(fmt, ".%d", precision);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
520 if (longflag)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
521 *fmt++ = 'l';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
522 else if (size_tflag) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
523 char *f = PY_FORMAT_SIZE_T;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
524 while (*f)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
525 *fmt++ = *f++;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
526 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
527 *fmt++ = c;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
528 *fmt = '\0';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
529 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
530
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
531 #define appendstring(string) {for (copy = string;*copy;) *s++ = *copy++;}
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
532
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
533 PyObject *
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
534 PyUnicode_FromFormatV(const char *format, va_list vargs)
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
535 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
536 va_list count;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
537 Py_ssize_t callcount = 0;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
538 PyObject **callresults = NULL;
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
539 PyObject **callresult = NULL;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
540 Py_ssize_t n = 0;
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
541 int width = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
542 int precision = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
543 int zeropad;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
544 const char* f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
545 Py_UNICODE *s;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
546 PyObject *string;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
547 /* used by sprintf */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
548 char buffer[21];
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
549 /* use abuffer instead of buffer, if we need more space
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
550 * (which can happen if there's a format specifier with width). */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
551 char *abuffer = NULL;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
552 char *realbuffer;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
553 Py_ssize_t abuffersize = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
554 char fmt[60]; /* should be enough for %0width.precisionld */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
555 const char *copy;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
556
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
557 #ifdef VA_LIST_IS_ARRAY
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
558 Py_MEMCPY(count, vargs, sizeof(va_list));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
559 #else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
560 #ifdef __va_copy
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
561 __va_copy(count, vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
562 #else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
563 count = vargs;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
564 #endif
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
565 #endif
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
566 /* step 1: count the number of %S/%R format specifications
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
567 * (we call PyObject_Unicode()/PyObject_Repr() for these objects
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
568 * once during step 3 and put the result in an array) */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
569 for (f = format; *f; f++) {
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
570 if (*f == '%' && (*(f+1)=='S' || *(f+1)=='R'))
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
571 ++callcount;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
572 }
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
573 /* step 2: allocate memory for the results of
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
574 * PyObject_Unicode()/PyObject_Repr() calls */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
575 if (callcount) {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
576 callresults = PyMem_Malloc(sizeof(PyObject *)*callcount);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
577 if (!callresults) {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
578 PyErr_NoMemory();
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
579 return NULL;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
580 }
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
581 callresult = callresults;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
582 }
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
583 /* step 3: figure out how large a buffer we need */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
584 for (f = format; *f; f++) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
585 if (*f == '%') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
586 const char* p = f;
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
587 width = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
588 while (isdigit(Py_CHARMASK(*f)))
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
589 width = (width*10) + *f++ - '0';
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
590 while (*++f && *f != '%' && !isalpha(Py_CHARMASK(*f)))
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
591 ;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
592
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
593 /* skip the 'l' or 'z' in {%ld, %zd, %lu, %zu} since
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
594 * they don't affect the amount of space we reserve.
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
595 */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
596 if ((*f == 'l' || *f == 'z') &&
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
597 (f[1] == 'd' || f[1] == 'u'))
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
598 ++f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
599
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
600 switch (*f) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
601 case 'c':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
602 (void)va_arg(count, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
603 /* fall through... */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
604 case '%':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
605 n++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
606 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
607 case 'd': case 'u': case 'i': case 'x':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
608 (void) va_arg(count, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
609 /* 20 bytes is enough to hold a 64-bit
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
610 integer. Decimal takes the most space.
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
611 This isn't enough for octal.
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
612 If a width is specified we need more
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
613 (which we allocate later). */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
614 if (width < 20)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
615 width = 20;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
616 n += width;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
617 if (abuffersize < width)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
618 abuffersize = width;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
619 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
620 case 's':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
621 n += strlen(va_arg(count, char*));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
622 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
623 case 'U':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
624 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
625 PyObject *obj = va_arg(count, PyObject *);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
626 assert(obj && PyUnicode_Check(obj));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
627 n += PyUnicode_GET_SIZE(obj);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
628 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
629 }
42047
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
630 case 'V':
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
631 {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
632 PyObject *obj = va_arg(count, PyObject *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
633 const char *str = va_arg(count, const char *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
634 assert(obj || str);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
635 assert(!obj || PyUnicode_Check(obj));
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
636 if (obj)
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
637 n += PyUnicode_GET_SIZE(obj);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
638 else
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
639 n += strlen(str);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
640 break;
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
641 }
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
642 case 'S':
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
643 {
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
644 PyObject *obj = va_arg(count, PyObject *);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
645 PyObject *str;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
646 assert(obj);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
647 str = PyObject_Unicode(obj);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
648 if (!str)
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
649 goto fail;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
650 n += PyUnicode_GET_SIZE(str);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
651 /* Remember the str and switch to the next slot */
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
652 *callresult++ = str;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
653 break;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
654 }
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
655 case 'R':
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
656 {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
657 PyObject *obj = va_arg(count, PyObject *);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
658 PyObject *repr;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
659 assert(obj);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
660 repr = PyObject_Repr(obj);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
661 if (!repr)
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
662 goto fail;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
663 n += PyUnicode_GET_SIZE(repr);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
664 /* Remember the repr and switch to the next slot */
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
665 *callresult++ = repr;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
666 break;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
667 }
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
668 case 'p':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
669 (void) va_arg(count, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
670 /* maximum 64-bit pointer representation:
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
671 * 0xffffffffffffffff
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
672 * so 19 characters is enough.
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
673 * XXX I count 18 -- what's the extra for?
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
674 */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
675 n += 19;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
676 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
677 default:
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
678 /* if we stumble upon an unknown
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
679 formatting code, copy the rest of
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
680 the format string to the output
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
681 string. (we cannot just skip the
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
682 code, since there's no way to know
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
683 what's in the argument list) */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
684 n += strlen(p);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
685 goto expand;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
686 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
687 } else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
688 n++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
689 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
690 expand:
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
691 if (abuffersize > 20) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
692 abuffer = PyMem_Malloc(abuffersize);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
693 if (!abuffer) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
694 PyErr_NoMemory();
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
695 goto fail;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
696 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
697 realbuffer = abuffer;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
698 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
699 else
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
700 realbuffer = buffer;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
701 /* step 4: fill the buffer */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
702 /* Since we've analyzed how much space we need for the worst case,
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
703 we don't have to resize the string.
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
704 There can be no errors beyond this point. */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
705 string = PyUnicode_FromUnicode(NULL, n);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
706 if (!string)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
707 goto fail;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
708
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
709 s = PyUnicode_AS_UNICODE(string);
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
710 callresult = callresults;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
711
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
712 for (f = format; *f; f++) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
713 if (*f == '%') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
714 const char* p = f++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
715 int longflag = 0;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
716 int size_tflag = 0;
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
717 zeropad = (*f == '0');
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
718 /* parse the width.precision part */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
719 width = 0;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
720 while (isdigit(Py_CHARMASK(*f)))
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
721 width = (width*10) + *f++ - '0';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
722 precision = 0;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
723 if (*f == '.') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
724 f++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
725 while (isdigit(Py_CHARMASK(*f)))
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
726 precision = (precision*10) + *f++ - '0';
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
727 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
728 /* handle the long flag, but only for %ld and %lu.
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
729 others can be added when necessary. */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
730 if (*f == 'l' && (f[1] == 'd' || f[1] == 'u')) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
731 longflag = 1;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
732 ++f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
733 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
734 /* handle the size_t flag. */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
735 if (*f == 'z' && (f[1] == 'd' || f[1] == 'u')) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
736 size_tflag = 1;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
737 ++f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
738 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
739
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
740 switch (*f) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
741 case 'c':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
742 *s++ = va_arg(vargs, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
743 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
744 case 'd':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
745 makefmt(fmt, longflag, size_tflag, zeropad, width, precision, 'd');
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
746 if (longflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
747 sprintf(realbuffer, fmt, va_arg(vargs, long));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
748 else if (size_tflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
749 sprintf(realbuffer, fmt, va_arg(vargs, Py_ssize_t));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
750 else
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
751 sprintf(realbuffer, fmt, va_arg(vargs, int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
752 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
753 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
754 case 'u':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
755 makefmt(fmt, longflag, size_tflag, zeropad, width, precision, 'u');
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
756 if (longflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
757 sprintf(realbuffer, fmt, va_arg(vargs, unsigned long));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
758 else if (size_tflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
759 sprintf(realbuffer, fmt, va_arg(vargs, size_t));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
760 else
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
761 sprintf(realbuffer, fmt, va_arg(vargs, unsigned int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
762 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
763 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
764 case 'i':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
765 makefmt(fmt, 0, 0, zeropad, width, precision, 'i');
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
766 sprintf(realbuffer, fmt, va_arg(vargs, int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
767 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
768 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
769 case 'x':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
770 makefmt(fmt, 0, 0, zeropad, width, precision, 'x');
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
771 sprintf(realbuffer, fmt, va_arg(vargs, int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
772 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
773 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
774 case 's':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
775 p = va_arg(vargs, char*);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
776 appendstring(p);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
777 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
778 case 'U':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
779 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
780 PyObject *obj = va_arg(vargs, PyObject *);
41861
969868df1938 Simplify %U handling by using Py_UNICODE_COPY.
Walter Dörwald <walter@livinglogic.de>
parents: 41840
diff changeset
781 Py_ssize_t size = PyUnicode_GET_SIZE(obj);
969868df1938 Simplify %U handling by using Py_UNICODE_COPY.
Walter Dörwald <walter@livinglogic.de>
parents: 41840
diff changeset
782 Py_UNICODE_COPY(s, PyUnicode_AS_UNICODE(obj), size);
969868df1938 Simplify %U handling by using Py_UNICODE_COPY.
Walter Dörwald <walter@livinglogic.de>
parents: 41840
diff changeset
783 s += size;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
784 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
785 }
42047
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
786 case 'V':
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
787 {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
788 PyObject *obj = va_arg(vargs, PyObject *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
789 const char *str = va_arg(vargs, const char *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
790 if (obj) {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
791 Py_ssize_t size = PyUnicode_GET_SIZE(obj);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
792 Py_UNICODE_COPY(s, PyUnicode_AS_UNICODE(obj), size);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
793 s += size;
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
794 } else {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
795 appendstring(str);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
796 }
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
797 break;
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
798 }
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
799 case 'S':
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
800 case 'R':
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
801 {
42073
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
802 Py_UNICODE *ucopy;
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
803 Py_ssize_t usize;
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
804 Py_ssize_t upos;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
805 /* unused, since we already have the result */
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
806 (void) va_arg(vargs, PyObject *);
42073
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
807 ucopy = PyUnicode_AS_UNICODE(*callresult);
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
808 usize = PyUnicode_GET_SIZE(*callresult);
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
809 for (upos = 0; upos<usize;)
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
810 *s++ = ucopy[upos++];
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
811 /* We're done with the unicode()/repr() => forget it */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
812 Py_DECREF(*callresult);
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
813 /* switch to next unicode()/repr() result */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
814 ++callresult;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
815 break;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
816 }
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
817 case 'p':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
818 sprintf(buffer, "%p", va_arg(vargs, void*));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
819 /* %p is ill-defined: ensure leading 0x. */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
820 if (buffer[1] == 'X')
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
821 buffer[1] = 'x';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
822 else if (buffer[1] != 'x') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
823 memmove(buffer+2, buffer, strlen(buffer)+1);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
824 buffer[0] = '0';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
825 buffer[1] = 'x';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
826 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
827 appendstring(buffer);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
828 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
829 case '%':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
830 *s++ = '%';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
831 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
832 default:
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
833 appendstring(p);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
834 goto end;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
835 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
836 } else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
837 *s++ = *f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
838 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
839
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
840 end:
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
841 if (callresults)
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
842 PyMem_Free(callresults);
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
843 if (abuffer)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
844 PyMem_Free(abuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
845 _PyUnicode_Resize(&string, s - PyUnicode_AS_UNICODE(string));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
846 return string;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
847 fail:
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
848 if (callresults) {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
849 PyObject **callresult2 = callresults;
42291
3772103200ec Three patches by Amaury Forgeot d'Arc; SF patch# 1754484.
Guido van Rossum <guido@python.org>
parents: 42283
diff changeset
850 while (callresult2 < callresult) {
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
851 Py_DECREF(*callresult2);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
852 ++callresult2;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
853 }
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
854 PyMem_Free(callresults);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
855 }
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
856 if (abuffer)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
857 PyMem_Free(abuffer);
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
858 return NULL;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
859 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
860
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
861 #undef appendstring
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
862
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
863 PyObject *
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
864 PyUnicode_FromFormat(const char *format, ...)
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
865 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
866 PyObject* ret;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
867 va_list vargs;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
868
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
869 #ifdef HAVE_STDARG_PROTOTYPES
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
870 va_start(vargs, format);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
871 #else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
872 va_start(vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
873 #endif
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
874 ret = PyUnicode_FromFormatV(format, vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
875 va_end(vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
876 return ret;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
877 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
878
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
879 Py_ssize_t PyUnicode_AsWideChar(PyUnicodeObject *unicode,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
880 wchar_t *w,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
881 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
882 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
883 if (unicode == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
884 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
885 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
886 }
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
887
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
888 /* If possible, try to copy the 0-termination as well */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
889 if (size > PyUnicode_GET_SIZE(unicode))
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
890 size = PyUnicode_GET_SIZE(unicode) + 1;
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
891
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
892 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
893 memcpy(w, unicode->str, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
894 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
895 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
896 register Py_UNICODE *u;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
897 register Py_ssize_t i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
898 u = PyUnicode_AS_UNICODE(unicode);
33766
1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
Marc-André Lemburg <mal@egenix.com>
parents: 33499
diff changeset
899 for (i = size; i > 0; i--)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
900 *w++ = *u++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
901 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
902 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
903
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
904 if (size > PyUnicode_GET_SIZE(unicode))
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
905 return PyUnicode_GET_SIZE(unicode);
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
906 else
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
907 return size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
908 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
909
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
910 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
911
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
912 PyObject *PyUnicode_FromOrdinal(int ordinal)
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
913 {
42283
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
914 Py_UNICODE s[2];
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
915
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
916 if (ordinal < 0 || ordinal > 0x10ffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
917 PyErr_SetString(PyExc_ValueError,
42283
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
918 "chr() arg not in range(0x110000)");
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
919 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
920 }
42283
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
921
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
922 #ifndef Py_UNICODE_WIDE
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
923 if (ordinal > 0xffff) {
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
924 ordinal -= 0x10000;
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
925 s[0] = 0xD800 | (ordinal >> 10);
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
926 s[1] = 0xDC00 | (ordinal & 0x3FF);
18e5ae91bd29 Make chr() and ord() return/accept surrogate pairs in narrow builds.
Guido van Rossum <guido@python.org>
parents: 42208
diff changeset
927 return PyUnicode_FromUnicode(s, 2);
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
928 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
929 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
930
31873
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
931 s[0] = (Py_UNICODE)ordinal;
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
932 return PyUnicode_FromUnicode(s, 1);
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
933 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
934
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
935 PyObject *PyUnicode_FromObject(register PyObject *obj)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
936 {
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
937 /* XXX Perhaps we should make this API an alias of
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
938 PyObject_Unicode() instead ?! */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
939 if (PyUnicode_CheckExact(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
940 Py_INCREF(obj);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
941 return obj;
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
942 }
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
943 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
944 /* For a Unicode subtype that's not a Unicode object,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
945 return a true Unicode object with the same data. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
946 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(obj),
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
947 PyUnicode_GET_SIZE(obj));
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
948 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
949 return PyUnicode_FromEncodedObject(obj, NULL, "strict");
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
950 }
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
951
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
952 PyObject *PyUnicode_FromEncodedObject(register PyObject *obj,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
953 const char *encoding,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
954 const char *errors)
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
955 {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
956 const char *s = NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
957 Py_ssize_t len;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
958 PyObject *v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
959
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
960 if (obj == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
961 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
962 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
963 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
964
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
965 #if 0
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
966 /* For b/w compatibility we also accept Unicode objects provided
20563
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
967 that no encodings is given and then redirect to
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
968 PyObject_Unicode() which then applies the additional logic for
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
969 Unicode subclasses.
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
970
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
971 NOTE: This API should really only be used for object which
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
972 represent *encoded* Unicode !
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
973
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
974 */
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
975 if (PyUnicode_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
976 if (encoding) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
977 PyErr_SetString(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
978 "decoding Unicode is not supported");
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
979 return NULL;
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
980 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
981 return PyObject_Unicode(obj);
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
982 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
983 #else
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
984 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
985 PyErr_SetString(PyExc_TypeError,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
986 "decoding Unicode is not supported");
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
987 return NULL;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
988 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
989 #endif
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
990
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
991 /* Coerce object */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
992 if (PyString_Check(obj)) {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
993 s = PyString_AS_STRING(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
994 len = PyString_GET_SIZE(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
995 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
996 else if (PyObject_AsCharBuffer(obj, &s, &len)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
997 /* Overwrite the error message with something more useful in
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
998 case of a TypeError. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
999 if (PyErr_ExceptionMatches(PyExc_TypeError))
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
1000 PyErr_Format(PyExc_TypeError,
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1001 "coercing to Unicode: need string or buffer, "
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1002 "%.80s found",
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
1003 Py_Type(obj)->tp_name);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1004 goto onError;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
1005 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1006
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1007 /* Convert to Unicode */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1008 if (len == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1009 Py_INCREF(unicode_empty);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1010 v = (PyObject *)unicode_empty;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1011 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1012 else
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1013 v = PyUnicode_Decode(s, len, encoding, errors);
15952
1f92d63b20f0 This patch adds a new builtin unistr() which behaves like str()
Marc-André Lemburg <mal@egenix.com>
parents: 15910
diff changeset
1014
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1015 return v;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1016
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1017 onError:
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1018 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1019 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1020
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1021 PyObject *PyUnicode_Decode(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1022 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1023 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1024 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1025 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1026 PyObject *buffer = NULL, *unicode;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1027
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1028 if (encoding == NULL)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1029 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1030
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1031 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1032 if (strcmp(encoding, "utf-8") == 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1033 return PyUnicode_DecodeUTF8(s, size, errors);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1034 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1035 return PyUnicode_DecodeLatin1(s, size, errors);
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1036 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1037 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1038 return PyUnicode_DecodeMBCS(s, size, errors);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1039 #endif
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1040 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1041 return PyUnicode_DecodeASCII(s, size, errors);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1042
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1043 /* Decode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1044 buffer = PyBuffer_FromMemory((void *)s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1045 if (buffer == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1046 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1047 unicode = PyCodec_Decode(buffer, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1048 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1049 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1050 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1051 PyErr_Format(PyExc_TypeError,
12320
5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format
Guido van Rossum <guido@python.org>
parents: 12275
diff changeset
1052 "decoder did not return an unicode object (type=%.400s)",
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
1053 Py_Type(unicode)->tp_name);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1054 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1055 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1056 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1057 Py_DECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1058 return unicode;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1059
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1060 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1061 Py_XDECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1062 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1063 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1064
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1065 PyObject *PyUnicode_AsDecodedObject(PyObject *unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1066 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1067 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1068 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1069 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1070
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1071 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1072 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1073 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1074 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1075
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1076 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1077 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1078
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1079 /* Decode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1080 v = PyCodec_Decode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1081 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1082 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1083 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1084
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1085 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1086 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1087 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1088
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1089 PyObject *PyUnicode_Encode(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1090 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1091 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1092 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1093 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1094 PyObject *v, *unicode;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1095
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1096 unicode = PyUnicode_FromUnicode(s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1097 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1098 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1099 v = PyUnicode_AsEncodedString(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1100 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1101 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1102 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1103
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1104 PyObject *PyUnicode_AsEncodedObject(PyObject *unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1105 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1106 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1107 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1108 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1109
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1110 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1111 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1112 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1113 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1114
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1115 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1116 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1117
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1118 /* Encode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1119 v = PyCodec_Encode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1120 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1121 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1122 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1123
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1124 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1125 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1126 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1127
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1128 PyObject *PyUnicode_AsEncodedString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1129 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1130 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1131 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1132 PyObject *v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1133
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1134 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1135 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1136 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1137 }
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1138
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1139 if (encoding == NULL)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1140 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1141
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1142 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1143 if (errors == NULL) {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1144 if (strcmp(encoding, "utf-8") == 0)
17995
837825a779c1 fix bogus indentation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17887
diff changeset
1145 return PyUnicode_AsUTF8String(unicode);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1146 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1147 return PyUnicode_AsLatin1String(unicode);
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1148 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1149 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1150 return PyUnicode_AsMBCSString(unicode);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1151 #endif
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1152 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1153 return PyUnicode_AsASCIIString(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1154 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1155
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1156 /* Encode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1157 v = PyCodec_Encode(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1158 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1159 goto onError;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1160 if (!PyBytes_Check(v)) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1161 if (PyString_Check(v)) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1162 /* Old codec, turn it into bytes */
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1163 PyObject *b = PyBytes_FromObject(v);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1164 Py_DECREF(v);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1165 return b;
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1166 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1167 PyErr_Format(PyExc_TypeError,
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1168 "encoder did not return a bytes object "
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1169 "(type=%.400s, encoding=%.20s, errors=%.20s)",
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1170 v->ob_type->tp_name,
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1171 encoding ? encoding : "NULL",
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1172 errors ? errors : "NULL");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1173 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1174 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1175 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1176 return v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1177
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1178 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1179 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1180 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1181
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1182 PyObject *_PyUnicode_AsDefaultEncodedString(PyObject *unicode,
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1183 const char *errors)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1184 {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1185 PyObject *v = ((PyUnicodeObject *)unicode)->defenc;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1186 PyObject *b;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1187 if (v)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1188 return v;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1189 if (errors != NULL)
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1190 Py_FatalError("non-NULL encoding in _PyUnicode_AsDefaultEncodedString");
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1191 if (errors == NULL) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1192 b = PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1193 PyUnicode_GET_SIZE(unicode),
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1194 NULL);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1195 }
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1196 else {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1197 b = PyUnicode_AsEncodedString(unicode, NULL, errors);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1198 }
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1199 if (!b)
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1200 return NULL;
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1201 v = PyString_FromStringAndSize(PyBytes_AsString(b),
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1202 PyBytes_Size(b));
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1203 Py_DECREF(b);
42208
ca3cfb34e3c2 Fixed a refcount leak in _PyUnicode_AsDefaultEncodedString().
Guido van Rossum <guido@python.org>
parents: 42184
diff changeset
1204 ((PyUnicodeObject *)unicode)->defenc = v;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1205 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1206 }
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1207
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1208 char*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1209 PyUnicode_AsString(PyObject *unicode)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1210 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1211 assert(PyUnicode_Check(unicode));
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1212 unicode = _PyUnicode_AsDefaultEncodedString(unicode, NULL);
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1213 if (!unicode)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1214 return NULL;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1215 return PyString_AsString(unicode);
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1216 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1217
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1218 Py_UNICODE *PyUnicode_AsUnicode(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1219 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1220 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1221 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1222 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1223 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1224 return PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1226 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1227 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1228 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1229
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1230 Py_ssize_t PyUnicode_GetSize(PyObject *unicode)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1231 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1232 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1233 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1234 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1235 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1236 return PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1237
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1238 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1239 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1240 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1241
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
1242 const char *PyUnicode_GetDefaultEncoding(void)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1243 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1244 return unicode_default_encoding;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1245 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1246
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1247 int PyUnicode_SetDefaultEncoding(const char *encoding)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1248 {
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1249 if (strcmp(encoding, unicode_default_encoding) != 0) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1250 PyErr_Format(PyExc_ValueError,
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1251 "Can only set default encoding to %s",
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1252 unicode_default_encoding);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1253 return -1;
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1254 }
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1255 return 0;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1256 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1257
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1258 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1259 build arguments, call the callback and check the arguments,
35806
44fe157255bc fix typos, mostly in comments
Fred Drake <fdrake@acm.org>
parents: 35758
diff changeset
1260 if no exception occurred, copy the replacement to the output
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1261 and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1262 return 0 on success, -1 on error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1263 */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1264
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1265 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1266 int unicode_decode_call_errorhandler(const char *errors, PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1267 const char *encoding, const char *reason,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1268 const char **input, const char **inend, Py_ssize_t *startinpos, Py_ssize_t *endinpos, PyObject **exceptionObject, const char **inptr,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1269 PyObject **output, Py_ssize_t *outpos, Py_UNICODE **outptr)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1270 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1271 static char *argparse = "O!n;decoding error handler must return (unicode, int) tuple";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1272
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1273 PyObject *restuple = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1274 PyObject *repunicode = NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1275 Py_ssize_t outsize = PyUnicode_GET_SIZE(*output);
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1276 Py_ssize_t insize;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1277 Py_ssize_t requiredsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1278 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1279 Py_UNICODE *repptr;
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1280 PyObject *inputobj = NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1281 Py_ssize_t repsize;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1282 int res = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1283
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1284 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1285 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1286 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1287 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1288 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1289
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1290 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1291 *exceptionObject = PyUnicodeDecodeError_Create(
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1292 encoding, *input, *inend-*input, *startinpos, *endinpos, reason);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1293 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1294 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1295 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1296 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1297 if (PyUnicodeDecodeError_SetStart(*exceptionObject, *startinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1298 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1299 if (PyUnicodeDecodeError_SetEnd(*exceptionObject, *endinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1300 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1301 if (PyUnicodeDecodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1302 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1303 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1304
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1305 restuple = PyObject_CallFunctionObjArgs(*errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1306 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1307 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1308 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1309 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1310 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1311 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1312 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type, &repunicode, &newpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1313 goto onError;
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1314
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1315 /* Copy back the bytes variables, which might have been modified by the
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1316 callback */
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1317 inputobj = PyUnicodeDecodeError_GetObject(*exceptionObject);
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1318 if (!inputobj)
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1319 goto onError;
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1320 if (!PyBytes_Check(inputobj)) {
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1321 PyErr_Format(PyExc_TypeError, "exception attribute object must be bytes");
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1322 }
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1323 *input = PyBytes_AS_STRING(inputobj);
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1324 insize = PyBytes_GET_SIZE(inputobj);
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1325 *inend = *input + insize;
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1326
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1327 if (newpos<0)
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1328 newpos = insize+newpos;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1329 if (newpos<0 || newpos>insize) {
36510
0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat.
Martin v. Löwis <martin@v.loewis.de>
parents: 36508
diff changeset
1330 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", newpos);
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1331 goto onError;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1332 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1333
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1334 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1335 have+the replacement+the rest of the string (starting
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1336 at the new input position), so we won't have to check space
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1337 when there are no errors in the rest of the string) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1338 repptr = PyUnicode_AS_UNICODE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1339 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1340 requiredsize = *outpos + repsize + insize-newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1341 if (requiredsize > outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1342 if (requiredsize<2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1343 requiredsize = 2*outsize;
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1344 if (PyUnicode_Resize(output, requiredsize) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1345 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1346 *outptr = PyUnicode_AS_UNICODE(*output) + *outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1347 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1348 *endinpos = newpos;
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1349 *inptr = *input + newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1350 Py_UNICODE_COPY(*outptr, repptr, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1351 *outptr += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1352 *outpos += repsize;
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1353
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1354 /* we made it! */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1355 res = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1356
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1357 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1358 Py_XDECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1359 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1360 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1361
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1362 /* --- UTF-7 Codec -------------------------------------------------------- */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1363
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1364 /* see RFC2152 for details */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1365
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1366 static
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1367 char utf7_special[128] = {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1368 /* indicate whether a UTF-7 character is special i.e. cannot be directly
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1369 encoded:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1370 0 - not special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1371 1 - special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1372 2 - whitespace (optional)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1373 3 - RFC2152 Set O (optional) */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1374 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1375 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1376 2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1377 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1378 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1379 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1380 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1381 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1382
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1383 };
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1384
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1385 /* Note: The comparison (c) <= 0 is a trick to work-around gcc
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1386 warnings about the comparison always being false; since
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1387 utf7_special[0] is 1, we can safely make that one comparison
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1388 true */
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1389
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1390 #define SPECIAL(c, encodeO, encodeWS) \
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1391 ((c) > 127 || (c) <= 0 || utf7_special[(c)] == 1 || \
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1392 (encodeWS && (utf7_special[(c)] == 2)) || \
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1393 (encodeO && (utf7_special[(c)] == 3)))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1394
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1395 #define B64(n) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1396 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[(n) & 0x3f])
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1397 #define B64CHAR(c) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1398 (isalnum(c) || (c) == '+' || (c) == '/')
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1399 #define UB64(c) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1400 ((c) == '+' ? 62 : (c) == '/' ? 63 : (c) >= 'a' ? \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1401 (c) - 71 : (c) >= 'A' ? (c) - 65 : (c) + 4 )
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1402
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1403 #define ENCODE(out, ch, bits) \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1404 while (bits >= 6) { \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1405 *out++ = B64(ch >> (bits-6)); \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1406 bits -= 6; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1407 }
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1408
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1409 #define DECODE(out, ch, bits, surrogate) \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1410 while (bits >= 16) { \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1411 Py_UNICODE outCh = (Py_UNICODE) ((ch >> (bits-16)) & 0xffff); \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1412 bits -= 16; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1413 if (surrogate) { \
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1414 /* We have already generated an error for the high surrogate \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1415 so let's not bother seeing if the low surrogate is correct or not */ \
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1416 surrogate = 0; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1417 } else if (0xDC00 <= outCh && outCh <= 0xDFFF) { \
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1418 /* This is a surrogate pair. Unfortunately we can't represent \
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1419 it in a 16-bit character */ \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1420 surrogate = 1; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1421 errmsg = "code pairs are not supported"; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1422 goto utf7Error; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1423 } else { \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1424 *out++ = outCh; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1425 } \
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1426 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1427
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1428 PyObject *PyUnicode_DecodeUTF7(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1429 Py_ssize_t size,
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1430 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1431 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1432 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1433 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1434 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1435 Py_ssize_t outpos;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1436 const char *e;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1437 PyUnicodeObject *unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1438 Py_UNICODE *p;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1439 const char *errmsg = "";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1440 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1441 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1442 unsigned long charsleft = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1443 int surrogate = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1444 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1445 PyObject *exc = NULL;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1446
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1447 unicode = _PyUnicode_New(size);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1448 if (!unicode)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1449 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1450 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1451 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1452
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1453 p = unicode->str;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1454 e = s + size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1455
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1456 while (s < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1457 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1458 restart:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1459 ch = *s;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1460
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1461 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1462 if ((ch == '-') || !B64CHAR(ch)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1463 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1464 s++;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1465
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1466 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1467 if (bitsleft >= 6) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1468 /* The shift sequence has a partial character in it. If
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1469 bitsleft < 6 then we could just classify it as padding
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1470 but that is not the case here */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1471
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1472 errmsg = "partial character in shift sequence";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1473 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1474 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1475 /* According to RFC2152 the remaining bits should be zero. We
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1476 choose to signal an error/insert a replacement character
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1477 here so indicate the potential of a misencoded character. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1478
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1479 /* On x86, a << b == a << (b%32) so make sure that bitsleft != 0 */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1480 if (bitsleft && charsleft << (sizeof(charsleft) * 8 - bitsleft)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1481 errmsg = "non-zero padding bits in shift sequence";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1482 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1483 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1484
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1485 if (ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1486 if ((s < e) && (*(s) == '-')) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1487 *p++ = '-';
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1488 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1489 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1490 } else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1491 errmsg = "unexpected special character";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1492 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1493 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1494 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1495 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1496 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1497 charsleft = (charsleft << 6) | UB64(ch);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1498 bitsleft += 6;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1499 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1500 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1501 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1502 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1503 else if ( ch == '+' ) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1504 startinpos = s-starts;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1505 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1506 if (s < e && *s == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1507 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1508 *p++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1509 } else
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1510 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1511 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1512 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1513 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1514 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1515 else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1516 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1517 s++;
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1518 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1519 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1520 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1521 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1522 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1523 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1524 continue;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1525 utf7Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1526 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1527 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1528 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1529 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1530 "utf7", errmsg,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1531 &starts, &e, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1532 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1533 goto onError;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1534 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1535
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1536 if (inShift) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1537 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1538 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1539 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1540 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1541 "utf7", "unterminated shift sequence",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1542 &starts, &e, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1543 (PyObject **)&unicode, &outpos, &p))
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1544 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1545 if (s < e)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1546 goto restart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1547 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1548
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1549 if (_PyUnicode_Resize(&unicode, p - PyUnicode_AS_UNICODE(unicode)) < 0)
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1550 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1551
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1552 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1553 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1554 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1555
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1556 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1557 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1558 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1559 Py_DECREF(unicode);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1560 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1561 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1562
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1563
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1564 PyObject *PyUnicode_EncodeUTF7(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1565 Py_ssize_t size,
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1566 int encodeSetO,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1567 int encodeWhiteSpace,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1568 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1569 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1570 PyObject *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1571 /* It might be possible to tighten this worst case */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1572 Py_ssize_t cbAllocated = 5 * size;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1573 int inShift = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1574 Py_ssize_t i = 0;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1575 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1576 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1577 char * out;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1578 char * start;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1579
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1580 if (size == 0)
41624
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1581 return PyBytes_FromStringAndSize(NULL, 0);
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1582
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1583 v = PyBytes_FromStringAndSize(NULL, cbAllocated);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1584 if (v == NULL)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1585 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1586
41624
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1587 start = out = PyBytes_AS_STRING(v);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1588 for (;i < size; ++i) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1589 Py_UNICODE ch = s[i];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1590
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1591 if (!inShift) {
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1592 if (ch == '+') {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1593 *out++ = '+';
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1594 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1595 } else if (SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1596 charsleft = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1597 bitsleft = 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1598 *out++ = '+';
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1599 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1600 inShift = bitsleft > 0;
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1601 } else {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1602 *out++ = (char) ch;
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1603 }
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1604 } else {
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1605 if (!SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1606 *out++ = B64(charsleft << (6-bitsleft));
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1607 charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1608 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1609 /* Characters not in the BASE64 set implicitly unshift the sequence
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1610 so no '-' is required, except if the character is itself a '-' */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1611 if (B64CHAR(ch) || ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1612 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1613 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1614 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1615 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1616 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1617 bitsleft += 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1618 charsleft = (charsleft << 16) | ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1619 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1620
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1621 /* If the next character is special then we dont' need to terminate
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1622 the shift sequence. If the next character is not a BASE64 character
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1623 or '-' then the shift sequence will be terminated implicitly and we
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1624 don't have to insert a '-'. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1625
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1626 if (bitsleft == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1627 if (i + 1 < size) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1628 Py_UNICODE ch2 = s[i+1];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1629
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1630 if (SPECIAL(ch2, encodeSetO, encodeWhiteSpace)) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1631
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1632 } else if (B64CHAR(ch2) || ch2 == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1633 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1634 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1635 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1636 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1637 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1638
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1639 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1640 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1641 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1642 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1643 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1644 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1645 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1646 }
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1647 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1648 if (bitsleft) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1649 *out++= B64(charsleft << (6-bitsleft) );
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1650 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1651 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1652
41624
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1653 if (PyBytes_Resize(v, out - start)) {
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1654 Py_DECREF(v);
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1655 return NULL;
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1656 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1657 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1658 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1659
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1660 #undef SPECIAL
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1661 #undef B64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1662 #undef B64CHAR
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1663 #undef UB64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1664 #undef ENCODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1665 #undef DECODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1666
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1667 /* --- UTF-8 Codec -------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1668
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1669 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1670 char utf8_code_length[256] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1671 /* Map UTF-8 encoded prefix byte to sequence length. zero means
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1672 illegal prefix. see RFC 2279 for details */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1673 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1674 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1675 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1676 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1677 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1678 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1679 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1680 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1681 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1682 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1683 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1684 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1685 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1686 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1687 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1688 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1689 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1690
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1691 PyObject *PyUnicode_DecodeUTF8(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1692 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1693 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1694 {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1695 return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1696 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1697
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1698 PyObject *PyUnicode_DecodeUTF8Stateful(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1699 Py_ssize_t size,
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1700 const char *errors,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1701 Py_ssize_t *consumed)
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1702 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1703 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1704 int n;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1705 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1706 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1707 Py_ssize_t outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1708 const char *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1709 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1710 Py_UNICODE *p;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1711 const char *errmsg = "";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1712 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1713 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1714
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1715 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1716 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1717 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1718 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1719 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1720 if (size == 0) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1721 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1722 *consumed = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1723 return (PyObject *)unicode;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1724 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1725
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1726 /* Unpack UTF-8 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1727 p = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1728 e = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1729
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1730 while (s < e) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1731 Py_UCS4 ch = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1732
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1733 if (ch < 0x80) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1734 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1735 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1736 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1737 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1738
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1739 n = utf8_code_length[ch];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1740
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1741 if (s + n > e) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1742 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1743 break;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1744 else {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1745 errmsg = "unexpected end of data";
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1746 startinpos = s-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1747 endinpos = size;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1748 goto utf8Error;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1749 }
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1750 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1751
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1752 switch (n) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1753
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1754 case 0:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1755 errmsg = "unexpected code byte";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1756 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1757 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1758 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1759
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1760 case 1:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1761 errmsg = "internal error";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1762 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1763 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1764 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1765
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1766 case 2:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1767 if ((s[1] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1768 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1769 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1770 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1771 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1772 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1773 ch = ((s[0] & 0x1f) << 6) + (s[1] & 0x3f);
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1774 if (ch < 0x80) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1775 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1776 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1777 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1778 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1779 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1780 else
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1781 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1782 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1783
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1784 case 3:
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1785 if ((s[1] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1786 (s[2] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1787 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1788 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1789 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1790 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1791 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1792 ch = ((s[0] & 0x0f) << 12) + ((s[1] & 0x3f) << 6) + (s[2] & 0x3f);
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1793 if (ch < 0x0800) {
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1794 /* Note: UTF-8 encodings of surrogates are considered
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1795 legal UTF-8 sequences;
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1796
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1797 XXX For wide builds (UCS-4) we should probably try
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1798 to recombine the surrogates into a single code
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1799 unit.
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1800 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1801 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1802 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1803 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1804 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1805 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1806 else
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1807 *p++ = (Py_UNICODE)ch;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1808 break;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1809
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1810 case 4:
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1811 if ((s[1] & 0xc0) != 0x80 ||
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1812 (s[2] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1813 (s[3] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1814 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1815 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1816 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1817 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1818 }
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1819 ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1820 ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1821 /* validate and convert to UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1822 if ((ch < 0x10000) /* minimum value allowed for 4
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1823 byte encoding */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1824 || (ch > 0x10ffff)) /* maximum value allowed for
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1825 UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1826 {
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1827 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1828 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1829 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1830 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1831 }
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1832 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1833 *p++ = (Py_UNICODE)ch;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1834 #else
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1835 /* compute and append the two surrogates: */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1836
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1837 /* translate from 10000..10FFFF to 0..FFFF */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1838 ch -= 0x10000;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1839
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1840 /* high surrogate = top 10 bits added to D800 */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1841 *p++ = (Py_UNICODE)(0xD800 + (ch >> 10));
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1842
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1843 /* low surrogate = bottom 10 bits added to DC00 */
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
1844 *p++ = (Py_UNICODE)(0xDC00 + (ch & 0x03FF));
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1845 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1846 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1847
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1848 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1849 /* Other sizes are only needed for UCS-4 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1850 errmsg = "unsupported Unicode code range";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1851 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1852 endinpos = startinpos+n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1853 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1854 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1855 s += n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1856 continue;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1857
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1858 utf8Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1859 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1860 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1861 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1862 "utf8", errmsg,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
1863 &starts, &e, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1864 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1865 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1866 }
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1867 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1868 *consumed = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1869
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1870 /* Adjust length */
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1871 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1872 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1873
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1874 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1875 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1876 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1877
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1878 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1879 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1880 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1881 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1882 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1883 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1884
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1885 /* Allocation strategy: if the string is short, convert into a stack buffer
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1886 and allocate exactly as much space needed at the end. Else allocate the
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1887 maximum possible needed (4 result bytes per Unicode character), and return
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1888 the excess memory at the end.
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1889 */
23198
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1890 PyObject *
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1891 PyUnicode_EncodeUTF8(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1892 Py_ssize_t size,
23198
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1893 const char *errors)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1894 {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1895 #define MAX_SHORT_UNICHARS 300 /* largest size we'll do on the stack */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1896
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1897 Py_ssize_t i; /* index into s of next input byte */
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1898 PyObject *v; /* result string object */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1899 char *p; /* next free byte in output buffer */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1900 Py_ssize_t nallocated; /* number of result bytes allocated */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
1901 Py_ssize_t nneeded; /* number of result bytes needed */
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1902 char stackbuf[MAX_SHORT_UNICHARS * 4];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1903
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1904 assert(s != NULL);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1905 assert(size >= 0);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1906
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1907 if (size <= MAX_SHORT_UNICHARS) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1908 /* Write into the stack buffer; nallocated can't overflow.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1909 * At the end, we'll allocate exactly as much heap space as it
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1910 * turns out we need.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1911 */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1912 nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1913 v = NULL; /* will allocate after we're done */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1914 p = stackbuf;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1915 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1916 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1917 /* Overallocate on the heap, and give the excess back at the end. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1918 nallocated = size * 4;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1919 if (nallocated / 4 != size) /* overflow! */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1920 return PyErr_NoMemory();
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1921 v = PyBytes_FromStringAndSize(NULL, nallocated);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1922 if (v == NULL)
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1923 return NULL;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1924 p = PyBytes_AS_STRING(v);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1925 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1926
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1927 for (i = 0; i < size;) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1928 Py_UCS4 ch = s[i++];
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1929
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1930 if (ch < 0x80)
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1931 /* Encode ASCII */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1932 *p++ = (char) ch;
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1933
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1934 else if (ch < 0x0800) {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1935 /* Encode Latin-1 */
21884
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1936 *p++ = (char)(0xc0 | (ch >> 6));
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1937 *p++ = (char)(0x80 | (ch & 0x3f));
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1938 }
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1939 else {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1940 /* Encode UCS2 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1941 if (ch < 0x10000) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1942 /* Special case: check for high surrogate */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1943 if (0xD800 <= ch && ch <= 0xDBFF && i != size) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1944 Py_UCS4 ch2 = s[i];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1945 /* Check for low surrogate and combine the two to
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1946 form a UCS4 value */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1947 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1948 ch = ((ch - 0xD800) << 10 | (ch2 - 0xDC00)) + 0x10000;
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1949 i++;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1950 goto encodeUCS4;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1951 }
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1952 /* Fall through: handles isolated high surrogates */
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1953 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1954 *p++ = (char)(0xe0 | (ch >> 12));
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1955 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1956 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1957 continue;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1958 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1959 encodeUCS4:
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1960 /* Encode UCS4 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1961 *p++ = (char)(0xf0 | (ch >> 18));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1962 *p++ = (char)(0x80 | ((ch >> 12) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1963 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1964 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1965 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1966 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1967
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1968 if (v == NULL) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1969 /* This was stack allocated. */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
1970 nneeded = p - stackbuf;
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1971 assert(nneeded <= nallocated);
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1972 v = PyBytes_FromStringAndSize(stackbuf, nneeded);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1973 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1974 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1975 /* Cut back to size actually needed. */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1976 nneeded = p - PyBytes_AS_STRING(v);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1977 assert(nneeded <= nallocated);
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1978 PyBytes_Resize(v, nneeded);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1979 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1980 return v;
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1981
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1982 #undef MAX_SHORT_UNICHARS
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1983 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1984
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1985 PyObject *PyUnicode_AsUTF8String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1986 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1987 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1988 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1989 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1990 }
14059
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1991 return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1992 PyUnicode_GET_SIZE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1993 NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1994 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1995
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1996 /* --- UTF-16 Codec ------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1997
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1998 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1999 PyUnicode_DecodeUTF16(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2000 Py_ssize_t size,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2001 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2002 int *byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2003 {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2004 return PyUnicode_DecodeUTF16Stateful(s, size, errors, byteorder, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2005 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2006
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2007 PyObject *
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2008 PyUnicode_DecodeUTF16Stateful(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2009 Py_ssize_t size,
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2010 const char *errors,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2011 int *byteorder,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2012 Py_ssize_t *consumed)
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2013 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2014 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2015 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2016 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2017 Py_ssize_t outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2018 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2019 Py_UNICODE *p;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2020 const unsigned char *q, *e;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2021 int bo = 0; /* assume native ordering by default */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2022 const char *errmsg = "";
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2023 /* Offsets from q for retrieving byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2024 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2025 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2026 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2027 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2028 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2029 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2030 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2031
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2032 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2033 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2034 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2035 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2036 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2037 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2038 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2039
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2040 /* Unpack UTF-16 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2041 p = unicode->str;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2042 q = (unsigned char *)s;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2043 e = q + size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2044
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2045 if (byteorder)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2046 bo = *byteorder;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2047
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2048 /* Check for BOM marks (U+FEFF) in the input and adjust current
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2049 byte order setting accordingly. In native mode, the leading BOM
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2050 mark is skipped, in all other modes, it is copied to the output
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2051 stream as-is (giving a ZWNBSP character). */
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2052 if (bo == 0) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2053 if (size >= 2) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2054 const Py_UNICODE bom = (q[ihi] << 8) | q[ilo];
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2055 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2056 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2057 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2058 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2059 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2060 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2061 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2062 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2063 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2064 #else
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2065 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2066 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2067 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2068 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2069 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2070 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2071 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2072 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2073 #endif
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2074 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2075 }
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2076
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2077 if (bo == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2078 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2079 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2080 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2081 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2082 else if (bo == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2083 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2084 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2085 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2086 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2087
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2088 while (q < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2089 Py_UNICODE ch;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2090 /* remaining bytes at the end? (size should be even) */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2091 if (e-q<2) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2092 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2093 break;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2094 errmsg = "truncated data";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2095 startinpos = ((const char *)q)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2096 endinpos = ((const char *)e)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2097 goto utf16Error;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2098 /* The remaining input chars are ignored if the callback
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2099 chooses to skip the input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2100 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2101 ch = (q[ihi] << 8) | q[ilo];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2102
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2103 q += 2;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2104
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2105 if (ch < 0xD800 || ch > 0xDFFF) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2106 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2107 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2108 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2109
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2110 /* UTF-16 code pair: */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2111 if (q >= e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2112 errmsg = "unexpected end of data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2113 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2114 endinpos = ((const char *)e)-starts;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2115 goto utf16Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2116 }
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
2117 if (0xD800 <= ch && ch <= 0xDBFF) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2118 Py_UNICODE ch2 = (q[ihi] << 8) | q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2119 q += 2;
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
2120 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
2121 #ifndef Py_UNICODE_WIDE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2122 *p++ = ch;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2123 *p++ = ch2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2124 #else
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2125 *p++ = (((ch & 0x3FF)<<10) | (ch2 & 0x3FF)) + 0x10000;
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2126 #endif
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2127 continue;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2128 }
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2129 else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2130 errmsg = "illegal UTF-16 surrogate";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2131 startinpos = (((const char *)q)-4)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2132 endinpos = startinpos+2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2133 goto utf16Error;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2134 }
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2135
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2136 }
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2137 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2138 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2139 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2140 /* Fall through to report the error */
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2141
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2142 utf16Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2143 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2144 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2145 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2146 "utf16", errmsg,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2147 &starts, (const char **)&e, &startinpos, &endinpos, &exc, (const char **)&q,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2148 (PyObject **)&unicode, &outpos, &p))
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2149 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2150 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2151
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2152 if (byteorder)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2153 *byteorder = bo;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2154
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2155 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2156 *consumed = (const char *)q-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2157
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2158 /* Adjust length */
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
2159 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2160 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2161
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2162 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2163 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2164 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2165
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2166 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2167 Py_DECREF(unicode);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2168 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2169 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2170 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2171 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2172
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2173 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2174 PyUnicode_EncodeUTF16(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2175 Py_ssize_t size,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2176 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2177 int byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2178 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2179 PyObject *v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2180 unsigned char *p;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2181 #ifdef Py_UNICODE_WIDE
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2182 int i, pairs;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2183 #else
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2184 const int pairs = 0;
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2185 #endif
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2186 /* Offsets from p for storing byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2187 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2188 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2189 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2190 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2191 #endif
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2192
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2193 #define STORECHAR(CH) \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2194 do { \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2195 p[ihi] = ((CH) >> 8) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2196 p[ilo] = (CH) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2197 p += 2; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2198 } while(0)
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2199
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2200 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2201 for (i = pairs = 0; i < size; i++)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2202 if (s[i] >= 0x10000)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2203 pairs++;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2204 #endif
41605
1162502a09da Change PyUnicode_EncodeUTF16() so that it returns
Walter Dörwald <walter@livinglogic.de>
parents: 41598
diff changeset
2205 v = PyBytes_FromStringAndSize(NULL,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2206 2 * (size + pairs + (byteorder == 0)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2207 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2208 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2209
41605
1162502a09da Change PyUnicode_EncodeUTF16() so that it returns
Walter Dörwald <walter@livinglogic.de>
parents: 41598
diff changeset
2210 p = (unsigned char *)PyBytes_AS_STRING(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2211 if (byteorder == 0)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2212 STORECHAR(0xFEFF);
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
2213 if (size == 0)
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2214 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2215
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2216 if (byteorder == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2217 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2218 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2219 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2220 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2221 else if (byteorder == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2222 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2223 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2224 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2225 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2226
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2227 while (size-- > 0) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2228 Py_UNICODE ch = *s++;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2229 Py_UNICODE ch2 = 0;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2230 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2231 if (ch >= 0x10000) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2232 ch2 = 0xDC00 | ((ch-0x10000) & 0x3FF);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2233 ch = 0xD800 | ((ch-0x10000) >> 10);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2234 }
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2235 #endif
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2236 STORECHAR(ch);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2237 if (ch2)
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2238 STORECHAR(ch2);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2239 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2240 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2241 #undef STORECHAR
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2242 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2243
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2244 PyObject *PyUnicode_AsUTF16String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2245 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2246 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2247 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2248 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2249 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2250 return PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2251 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2252 NULL,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2253 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2254 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2255
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2256 /* --- Unicode Escape Codec ----------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2257
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2258 static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL;
13063
eb243443b884 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13049
diff changeset
2259
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2260 PyObject *PyUnicode_DecodeUnicodeEscape(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2261 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2262 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2263 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2264 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2265 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2266 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2267 Py_ssize_t outpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2268 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2269 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2270 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2271 const char *end;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2272 char* message;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2273 Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2274 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2275 PyObject *exc = NULL;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2276
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2277 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2278 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2279 length after conversion to the true value.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2280 (but if the error callback returns a long replacement string
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2281 we'll have to allocate more space) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2282 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2283 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2284 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2285 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2286 return (PyObject *)v;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2287
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2288 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2289 end = s + size;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2290
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2291 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2292 unsigned char c;
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
2293 Py_UNICODE x;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2294 int digits;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2295
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2296 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2297 if (*s != '\\') {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2298 *p++ = (unsigned char) *s++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2299 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2300 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2301
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2302 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2303 /* \ - Escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2304 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2305 switch (*s++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2306
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2307 /* \x escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2308 case '\n': break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2309 case '\\': *p++ = '\\'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2310 case '\'': *p++ = '\''; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2311 case '\"': *p++ = '\"'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2312 case 'b': *p++ = '\b'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2313 case 'f': *p++ = '\014'; break; /* FF */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2314 case 't': *p++ = '\t'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2315 case 'n': *p++ = '\n'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2316 case 'r': *p++ = '\r'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2317 case 'v': *p++ = '\013'; break; /* VT */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2318 case 'a': *p++ = '\007'; break; /* BEL, not classic C */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2319
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2320 /* \OOO (octal) escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2321 case '0': case '1': case '2': case '3':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2322 case '4': case '5': case '6': case '7':
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2323 x = s[-1] - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2324 if ('0' <= *s && *s <= '7') {
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2325 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2326 if ('0' <= *s && *s <= '7')
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2327 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2328 }
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2329 *p++ = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2330 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2331
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2332 /* hex escapes */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2333 /* \xXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2334 case 'x':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2335 digits = 2;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2336 message = "truncated \\xXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2337 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2338
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2339 /* \uXXXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2340 case 'u':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2341 digits = 4;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2342 message = "truncated \\uXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2343 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2344
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2345 /* \UXXXXXXXX */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2346 case 'U':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2347 digits = 8;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2348 message = "truncated \\UXXXXXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2349 hexescape:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2350 chr = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2351 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2352 if (s+digits>end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2353 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2354 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2355 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2356 "unicodeescape", "end of string in escape sequence",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2357 &starts, &end, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2358 (PyObject **)&v, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2359 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2360 goto nextByte;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2361 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2362 for (i = 0; i < digits; ++i) {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2363 c = (unsigned char) s[i];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2364 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2365 endinpos = (s+i+1)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2366 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2367 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2368 "unicodeescape", message,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2369 &starts, &end, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2370 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2371 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2372 goto nextByte;
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2373 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2374 chr = (chr<<4) & ~0xF;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2375 if (c >= '0' && c <= '9')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2376 chr += c - '0';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2377 else if (c >= 'a' && c <= 'f')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2378 chr += 10 + c - 'a';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2379 else
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2380 chr += 10 + c - 'A';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2381 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2382 s += i;
30653
3b47d6057151 Fix for SF bug [ 817156 ] invalid \U escape gives 0=length unistr.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30441
diff changeset
2383 if (chr == 0xffffffff && PyErr_Occurred())
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2384 /* _decoding_error will have already written into the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2385 target buffer. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2386 break;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2387 store:
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2388 /* when we get here, chr is a 32-bit unicode character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2389 if (chr <= 0xffff)
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2390 /* UCS-2 character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2391 *p++ = (Py_UNICODE) chr;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2392 else if (chr <= 0x10ffff) {
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2393 /* UCS-4 character. Either store directly, or as
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2394 surrogate pair. */
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
2395 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2396 *p++ = chr;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2397 #else
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2398 chr -= 0x10000L;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2399 *p++ = 0xD800 + (Py_UNICODE) (chr >> 10);
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
2400 *p++ = 0xDC00 + (Py_UNICODE) (chr & 0x03FF);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2401 #endif
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2402 } else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2403 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2404 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2405 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2406 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2407 "unicodeescape", "illegal Unicode character",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2408 &starts, &end, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2409 (PyObject **)&v, &outpos, &p))
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2410 goto onError;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2411 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2412 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2413
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2414 /* \N{name} */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2415 case 'N':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2416 message = "malformed \\N character escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2417 if (ucnhash_CAPI == NULL) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2418 /* load the unicode data module */
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2419 PyObject *m, *api;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2420 m = PyImport_ImportModule("unicodedata");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2421 if (m == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2422 goto ucnhashError;
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2423 api = PyObject_GetAttrString(m, "ucnhash_CAPI");
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2424 Py_DECREF(m);
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2425 if (api == NULL)
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2426 goto ucnhashError;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2427 ucnhash_CAPI = (_PyUnicode_Name_CAPI *)PyCObject_AsVoidPtr(api);
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2428 Py_DECREF(api);
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2429 if (ucnhash_CAPI == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2430 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2431 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2432 if (*s == '{') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2433 const char *start = s+1;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2434 /* look for the closing brace */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2435 while (*s != '}' && s < end)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2436 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2437 if (s > start && s < end && *s == '}') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2438 /* found a name. look it up in the unicode database */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2439 message = "unknown Unicode character name";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2440 s++;
36924
48e0a5e39403 Update Unicode database to Unicode 4.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 36882
diff changeset
2441 if (ucnhash_CAPI->getcode(NULL, start, (int)(s-start-1), &chr))
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2442 goto store;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2443 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2444 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2445 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2446 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2447 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2448 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2449 "unicodeescape", message,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2450 &starts, &end, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2451 (PyObject **)&v, &outpos, &p))
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2452 goto onError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2453 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2454
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2455 default:
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2456 if (s > end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2457 message = "\\ at end of string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2458 s--;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2459 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2460 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2461 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2462 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2463 "unicodeescape", message,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2464 &starts, &end, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2465 (PyObject **)&v, &outpos, &p))
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2466 goto onError;
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2467 }
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2468 else {
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2469 *p++ = '\\';
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2470 *p++ = (unsigned char)s[-1];
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2471 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2472 break;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2473 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2474 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2475 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2476 }
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2477 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2478 goto onError;
30160
3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()).
Walter Dörwald <walter@livinglogic.de>
parents: 30149
diff changeset
2479 Py_XDECREF(errorHandler);
3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()).
Walter Dörwald <walter@livinglogic.de>
parents: 30149
diff changeset
2480 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2481 return (PyObject *)v;
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2482
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2483 ucnhashError:
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2484 PyErr_SetString(
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2485 PyExc_UnicodeError,
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2486 "\\N escapes not supported (can't load unicodedata module)"
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2487 );
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2488 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2489 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2490 Py_XDECREF(exc);
16064
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
2491 return NULL;
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
2492
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2493 onError:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2494 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2495 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2496 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2497 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2498 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2499
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2500 /* Return a Unicode-Escape string version of the Unicode object.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2501
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2502 If quotes is true, the string is enclosed in u"" or u'' quotes as
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2503 appropriate.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2504
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2505 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2506
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2507 Py_LOCAL_INLINE(const Py_UNICODE *) findchar(const Py_UNICODE *s,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2508 Py_ssize_t size,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2509 Py_UNICODE ch)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2510 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2511 /* like wcschr, but doesn't stop at NULL characters */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2512
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2513 while (size-- > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2514 if (*s == ch)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2515 return s;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2516 s++;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2517 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2518
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2519 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2520 }
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
2521
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2522 static const char *hexdigits = "0123456789abcdef";
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2523
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2524 PyObject *PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s,
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2525 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2526 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2527 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2528 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2529
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2530 /* XXX(nnorwitz): rather than over-allocating, it would be
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2531 better to choose a different scheme. Perhaps scan the
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2532 first N-chars of the string and allocate based on that size.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2533 */
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2534 /* Initial allocation is based on the longest-possible unichr
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2535 escape.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2536
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2537 In wide (UTF-32) builds '\U00xxxxxx' is 10 chars per source
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2538 unichr, so in this case it's the longest unichr escape. In
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2539 narrow (UTF-16) builds this is five chars per source unichr
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2540 since there are two unichrs in the surrogate pair, so in narrow
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2541 (UTF-16) builds it's not the longest unichr escape.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2542
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2543 In wide or narrow builds '\uxxxx' is 6 chars per source unichr,
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2544 so in the narrow (UTF-16) build case it's the longest unichr
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2545 escape.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2546 */
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2547
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2548 repr = PyBytes_FromStringAndSize(NULL,
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2549 #ifdef Py_UNICODE_WIDE
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2550 + 10*size
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2551 #else
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2552 + 6*size
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2553 #endif
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2554 + 1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2555 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2556 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2557
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2558 p = PyBytes_AS_STRING(repr);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2559
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2560 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2561 Py_UNICODE ch = *s++;
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2562
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2563 /* Escape backslashes */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2564 if (ch == '\\') {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2565 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2566 *p++ = (char) ch;
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2567 continue;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2568 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2569
18773
743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This
Guido van Rossum <guido@python.org>
parents: 18324
diff changeset
2570 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2571 /* Map 21-bit characters to '\U00xxxxxx' */
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2572 else if (ch >= 0x10000) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2573 *p++ = '\\';
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2574 *p++ = 'U';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2575 *p++ = hexdigits[(ch >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2576 *p++ = hexdigits[(ch >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2577 *p++ = hexdigits[(ch >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2578 *p++ = hexdigits[(ch >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2579 *p++ = hexdigits[(ch >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2580 *p++ = hexdigits[(ch >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2581 *p++ = hexdigits[(ch >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2582 *p++ = hexdigits[ch & 0x0000000F];
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2583 continue;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2584 }
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2585 #else
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2586 /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2587 else if (ch >= 0xD800 && ch < 0xDC00) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2588 Py_UNICODE ch2;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2589 Py_UCS4 ucs;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2590
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2591 ch2 = *s++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2592 size--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2593 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2594 ucs = (((ch & 0x03FF) << 10) | (ch2 & 0x03FF)) + 0x00010000;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2595 *p++ = '\\';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2596 *p++ = 'U';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2597 *p++ = hexdigits[(ucs >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2598 *p++ = hexdigits[(ucs >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2599 *p++ = hexdigits[(ucs >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2600 *p++ = hexdigits[(ucs >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2601 *p++ = hexdigits[(ucs >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2602 *p++ = hexdigits[(ucs >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2603 *p++ = hexdigits[(ucs >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2604 *p++ = hexdigits[ucs & 0x0000000F];
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2605 continue;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2606 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2607 /* Fall through: isolated surrogates are copied as-is */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2608 s--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2609 size++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2610 }
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2611 #endif
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2612
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2613 /* Map 16-bit characters to '\uxxxx' */
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2614 if (ch >= 256) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2615 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2616 *p++ = 'u';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2617 *p++ = hexdigits[(ch >> 12) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2618 *p++ = hexdigits[(ch >> 8) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2619 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2620 *p++ = hexdigits[ch & 0x000F];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2621 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2622
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2623 /* Map special whitespace to '\t', \n', '\r' */
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2624 else if (ch == '\t') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2625 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2626 *p++ = 't';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2627 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2628 else if (ch == '\n') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2629 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2630 *p++ = 'n';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2631 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2632 else if (ch == '\r') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2633 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2634 *p++ = 'r';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2635 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2636
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2637 /* Map non-printable US ASCII to '\xhh' */
21067
c9c4c523dcbf Fix for bug #485951: repr diff between string and unicode.
Marc-André Lemburg <mal@egenix.com>
parents: 21028
diff changeset
2638 else if (ch < ' ' || ch >= 0x7F) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2639 *p++ = '\\';
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2640 *p++ = 'x';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2641 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2642 *p++ = hexdigits[ch & 0x000F];
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2643 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2644
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2645 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2646 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2647 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2648 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2649
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2650 *p = '\0';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2651 if (PyBytes_Resize(repr, p - PyBytes_AS_STRING(repr))) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2652 Py_DECREF(repr);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2653 return NULL;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2654 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2655 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2656 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2657
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2658 PyObject *PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2659 {
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2660 PyObject *s, *result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2661 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2662 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2663 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2664 }
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2665 s = PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2666 PyUnicode_GET_SIZE(unicode));
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2667
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2668 if (!s)
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2669 return NULL;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2670 result = PyString_FromStringAndSize(PyBytes_AS_STRING(s),
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2671 PyBytes_GET_SIZE(s));
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2672 Py_DECREF(s);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2673 return result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2674 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2675
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2676 /* --- Raw Unicode Escape Codec ------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2677
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2678 PyObject *PyUnicode_DecodeRawUnicodeEscape(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2679 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2680 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2681 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2682 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2683 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2684 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2685 Py_ssize_t outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2686 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2687 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2688 const char *end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2689 const char *bs;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2690 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2691 PyObject *exc = NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2692
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2693 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2694 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2695 length after conversion to the true value. (But decoding error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2696 handler might have to resize the string) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2697 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2698 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2699 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2700 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2701 return (PyObject *)v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2702 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2703 end = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2704 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2705 unsigned char c;
22423
b523a470a0ce Do not insert characters for unicode-escape decoders if the error mode
Martin v. Löwis <martin@v.loewis.de>
parents: 22087
diff changeset
2706 Py_UCS4 x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2707 int i;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2708 int count;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2709
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2710 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2711 if (*s != '\\') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2712 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2713 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2714 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2715 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2716
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2717 /* \u-escapes are only interpreted iff the number of leading
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2718 backslashes if odd */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2719 bs = s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2720 for (;s < end;) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2721 if (*s != '\\')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2722 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2723 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2724 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2725 if (((s - bs) & 1) == 0 ||
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2726 s >= end ||
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2727 (*s != 'u' && *s != 'U')) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2728 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2729 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2730 p--;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2731 count = *s=='u' ? 4 : 8;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2732 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2733
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2734 /* \uXXXX with 4 hex digits, \Uxxxxxxxx with 8 */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2735 outpos = p-PyUnicode_AS_UNICODE(v);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2736 for (x = 0, i = 0; i < count; ++i, ++s) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2737 c = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2738 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2739 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2740 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2741 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2742 "rawunicodeescape", "truncated \\uXXXX",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2743 &starts, &end, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2744 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2745 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2746 goto nextByte;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2747 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2748 x = (x<<4) & ~0xF;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2749 if (c >= '0' && c <= '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2750 x += c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2751 else if (c >= 'a' && c <= 'f')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2752 x += 10 + c - 'a';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2753 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2754 x += 10 + c - 'A';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2755 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2756 #ifndef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2757 if (x > 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2758 if (unicode_decode_call_errorhandler(
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2759 errors, &errorHandler,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2760 "rawunicodeescape", "\\Uxxxxxxxx out of range",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2761 &starts, &end, &startinpos, &endinpos, &exc, &s,
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2762 (PyObject **)&v, &outpos, &p))
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2763 goto onError;
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2764 }
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2765 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2766 *p++ = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2767 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2768 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2769 }
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2770 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2771 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2772 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2773 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2774 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2775
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2776 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2777 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2778 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2779 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2780 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2781 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2782
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2783 PyObject *PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2784 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2785 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2786 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2787 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2788 char *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2789
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2790 #ifdef Py_UNICODE_WIDE
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2791 repr = PyBytes_FromStringAndSize(NULL, 10 * size);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2792 #else
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2793 repr = PyBytes_FromStringAndSize(NULL, 6 * size);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2794 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2795 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2796 return NULL;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2797 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2798 return repr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2799
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2800 p = q = PyBytes_AS_STRING(repr);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2801 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2802 Py_UNICODE ch = *s++;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2803 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2804 /* Map 32-bit characters to '\Uxxxxxxxx' */
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2805 if (ch >= 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2806 *p++ = '\\';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2807 *p++ = 'U';
41704
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2808 *p++ = hexdigits[(ch >> 28) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2809 *p++ = hexdigits[(ch >> 24) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2810 *p++ = hexdigits[(ch >> 20) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2811 *p++ = hexdigits[(ch >> 16) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2812 *p++ = hexdigits[(ch >> 12) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2813 *p++ = hexdigits[(ch >> 8) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2814 *p++ = hexdigits[(ch >> 4) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2815 *p++ = hexdigits[ch & 15];
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2816 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2817 else
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2818 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2819 /* Map 16-bit characters to '\uxxxx' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2820 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2821 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2822 *p++ = 'u';
41704
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2823 *p++ = hexdigits[(ch >> 12) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2824 *p++ = hexdigits[(ch >> 8) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2825 *p++ = hexdigits[(ch >> 4) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2826 *p++ = hexdigits[ch & 15];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2827 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2828 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2829 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2830 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2831 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2832 *p = '\0';
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2833 if (PyBytes_Resize(repr, p - q)) {
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2834 Py_DECREF(repr);
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2835 return NULL;
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2836 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2837 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2838 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2839
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2840 PyObject *PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2841 {
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2842 PyObject *s, *result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2843 if (!PyUnicode_Check(unicode)) {
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2844 PyErr_BadArgument();
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2845 return NULL;
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2846 }
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2847 s = PyUnicode_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2848 PyUnicode_GET_SIZE(unicode));
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2849
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2850 if (!s)
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2851 return NULL;
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2852 result = PyString_FromStringAndSize(PyBytes_AS_STRING(s),
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2853 PyBytes_GET_SIZE(s));
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2854 Py_DECREF(s);
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2855 return result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2856 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2857
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2858 /* --- Unicode Internal Codec ------------------------------------------- */
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2859
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2860 PyObject *_PyUnicode_DecodeUnicodeInternal(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2861 Py_ssize_t size,
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2862 const char *errors)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2863 {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2864 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2865 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2866 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2867 Py_ssize_t outpos;
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2868 PyUnicodeObject *v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2869 Py_UNICODE *p;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2870 const char *end;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2871 const char *reason;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2872 PyObject *errorHandler = NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2873 PyObject *exc = NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2874
36236
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2875 #ifdef Py_UNICODE_WIDE
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2876 Py_UNICODE unimax = PyUnicode_GetMax();
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2877 #endif
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2878
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2879 /* XXX overflow detection missing */
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2880 v = _PyUnicode_New((size+Py_UNICODE_SIZE-1)/ Py_UNICODE_SIZE);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2881 if (v == NULL)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2882 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2883 if (PyUnicode_GetSize((PyObject *)v) == 0)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2884 return (PyObject *)v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2885 p = PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2886 end = s + size;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2887
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2888 while (s < end) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2889 memcpy(p, s, sizeof(Py_UNICODE));
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2890 /* We have to sanity check the raw data, otherwise doom looms for
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2891 some malformed UCS-4 data. */
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2892 if (
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2893 #ifdef Py_UNICODE_WIDE
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2894 *p > unimax || *p < 0 ||
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2895 #endif
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2896 end-s < Py_UNICODE_SIZE
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2897 )
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2898 {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2899 startinpos = s - starts;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2900 if (end-s < Py_UNICODE_SIZE) {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2901 endinpos = end-starts;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2902 reason = "truncated input";
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2903 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2904 else {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2905 endinpos = s - starts + Py_UNICODE_SIZE;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2906 reason = "illegal code point (> 0x10FFFF)";
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2907 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2908 outpos = p - PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2909 if (unicode_decode_call_errorhandler(
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2910 errors, &errorHandler,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2911 "unicode_internal", reason,
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
2912 &starts, &end, &startinpos, &endinpos, &exc, &s,
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2913 (PyObject **)&v, &outpos, &p)) {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2914 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2915 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2916 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2917 else {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2918 p++;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2919 s += Py_UNICODE_SIZE;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2920 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2921 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2922
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2923 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2924 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2925 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2926 Py_XDECREF(exc);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2927 return (PyObject *)v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2928
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2929 onError:
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2930 Py_XDECREF(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2931 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2932 Py_XDECREF(exc);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2933 return NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2934 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2935
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2936 /* --- Latin-1 Codec ------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2937
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2938 PyObject *PyUnicode_DecodeLatin1(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2939 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2940 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2941 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2942 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2943 Py_UNICODE *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2944
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2945 /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2946 if (size == 1) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2947 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2948 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2949 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2950
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2951 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2952 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2953 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2954 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2955 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2956 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2957 while (size-- > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2958 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2959 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2960
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2961 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2962 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2963 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2964 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2965
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2966 /* create or adjust a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2967 static void make_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2968 const char *encoding,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2969 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2970 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2971 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2972 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2973 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2974 *exceptionObject = PyUnicodeEncodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2975 encoding, unicode, size, startpos, endpos, reason);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2976 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2977 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2978 if (PyUnicodeEncodeError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2979 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2980 if (PyUnicodeEncodeError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2981 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2982 if (PyUnicodeEncodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2983 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2984 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2985 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2986 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2987 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2988 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2989 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2990
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2991 /* raises a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2992 static void raise_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2993 const char *encoding,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2994 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2995 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2996 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2997 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2998 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2999 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3000 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3001 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3002 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3003
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3004 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3005 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3006 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3007 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3008 static PyObject *unicode_encode_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3009 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3010 const char *encoding, const char *reason,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3011 const Py_UNICODE *unicode, Py_ssize_t size, PyObject **exceptionObject,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3012 Py_ssize_t startpos, Py_ssize_t endpos,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3013 Py_ssize_t *newpos)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3014 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3015 static char *argparse = "O!n;encoding error handler must return (unicode, int) tuple";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3016
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3017 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3018 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3019
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3020 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3021 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3022 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3023 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3024 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3025
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3026 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3027 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3028 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3029 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3030
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3031 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3032 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3033 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3034 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3035 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3036 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3037 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3038 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3039 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3040 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3041 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3042 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3043 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3044 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3045 if (*newpos<0)
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3046 *newpos = size+*newpos;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3047 if (*newpos<0 || *newpos>size) {
36510
0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat.
Martin v. Löwis <martin@v.loewis.de>
parents: 36508
diff changeset
3048 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", *newpos);
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3049 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3050 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3051 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3052 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3053 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3054 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3055 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3056
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3057 static PyObject *unicode_encode_ucs1(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3058 Py_ssize_t size,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3059 const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3060 int limit)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3061 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3062 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3063 PyObject *res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3064 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3065 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3066 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3067 /* pointer to the beginning of the unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3068 /* const Py_UNICODE *badp = NULL; */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3069 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3070 char *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3071 /* current output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3072 Py_ssize_t respos = 0;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3073 Py_ssize_t ressize;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3074 const char *encoding = (limit == 256) ? "latin-1" : "ascii";
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3075 const char *reason = (limit == 256) ? "ordinal not in range(256)" : "ordinal not in range(128)";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3076 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3077 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3078 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3079 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3080 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3081
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3082 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3083 replacements, if we need more, we'll resize */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3084 res = PyBytes_FromStringAndSize(NULL, size);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3085 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3086 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3087 if (size == 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3088 return res;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3089 str = PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3090 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3092 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3093 Py_UNICODE c = *p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3094
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3095 /* can we encode this? */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3096 if (c<limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3097 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3098 *str++ = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3099 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3100 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3101 else {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3102 Py_ssize_t unicodepos = p-startp;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3103 Py_ssize_t requiredsize;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3104 PyObject *repunicode;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3105 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3106 Py_ssize_t newpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3107 Py_ssize_t respos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3108 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3109 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3110 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3111 const Py_UNICODE *collend = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3112 /* find all unecodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3113 while ((collend < endp) && ((*collend)>=limit))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3114 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3115 /* cache callback name lookup (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3116 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3117 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3118 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3119 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3120 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3121 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3122 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3123 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3124 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3125 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3126 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3127 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3128 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3129 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3130 raise_encode_exception(&exc, encoding, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3131 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3132 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3133 while (collstart++<collend)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3134 *str++ = '?'; /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3135 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3136 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3137 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3138 case 4: /* xmlcharrefreplace */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3139 respos = str - PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3140 /* determine replacement size (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3141 for (p = collstart, repsize = 0; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3142 if (*p<10)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3143 repsize += 2+1+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3144 else if (*p<100)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3145 repsize += 2+2+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3146 else if (*p<1000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3147 repsize += 2+3+1;
31248
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
3148 else if (*p<10000)
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
3149 repsize += 2+4+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3150 #ifndef Py_UNICODE_WIDE
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3151 else
31248
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
3152 repsize += 2+5+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3153 #else
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3154 else if (*p<100000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3155 repsize += 2+5+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3156 else if (*p<1000000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3157 repsize += 2+6+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3158 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3159 repsize += 2+7+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3160 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3161 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3162 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3163 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3164 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3165 requiredsize = 2*ressize;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3166 if (PyBytes_Resize(res, requiredsize))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3167 goto onError;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3168 str = PyBytes_AS_STRING(res) + respos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3169 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3170 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3171 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3172 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3173 str += sprintf(str, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3174 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3175 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3176 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3177 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3178 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3179 encoding, reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3180 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3181 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3182 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3183 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3184 have+the replacement+the rest of the string, so
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3185 we won't have to check space for encodable characters) */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3186 respos = str - PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3187 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3188 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3189 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3190 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3191 requiredsize = 2*ressize;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3192 if (PyBytes_Resize(res, requiredsize)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3193 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3194 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3195 }
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3196 str = PyBytes_AS_STRING(res) + respos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3197 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3198 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3199 /* check if there is anything unencodable in the replacement
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3200 and copy it to the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3201 for (uni2 = PyUnicode_AS_UNICODE(repunicode);repsize-->0; ++uni2, ++str) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3202 c = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3203 if (c >= limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3204 raise_encode_exception(&exc, encoding, startp, size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3205 unicodepos, unicodepos+1, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3206 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3207 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3208 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3209 *str = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3210 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3211 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3212 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3213 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3214 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3215 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3216 /* Resize if we allocated to much */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3217 respos = str - PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3218 if (respos<ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3219 /* If this falls res will be NULL */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3220 PyBytes_Resize(res, respos);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3221 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3222 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3223 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3224
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3225 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3226 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3227 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3228 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3229 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3230 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3231
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3232 PyObject *PyUnicode_EncodeLatin1(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3233 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3234 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3235 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3236 return unicode_encode_ucs1(p, size, errors, 256);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3237 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3239 PyObject *PyUnicode_AsLatin1String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3240 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3241 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3242 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3243 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3244 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3245 return PyUnicode_EncodeLatin1(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3246 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3247 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3248 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3249
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3250 /* --- 7-bit ASCII Codec -------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3251
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3252 PyObject *PyUnicode_DecodeASCII(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3253 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3254 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3255 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3256 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3257 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3258 Py_UNICODE *p;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3259 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3260 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3261 Py_ssize_t outpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3262 const char *e;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3263 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3264 PyObject *exc = NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3265
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3266 /* ASCII is equivalent to the first 128 ordinals in Unicode. */
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3267 if (size == 1 && *(unsigned char*)s < 128) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3268 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3269 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3270 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3271
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3272 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3273 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3274 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3275 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3276 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3277 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3278 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3279 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3280 register unsigned char c = (unsigned char)*s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3281 if (c < 128) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3282 *p++ = c;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3283 ++s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3284 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3285 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3286 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3287 endinpos = startinpos + 1;
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
3288 outpos = p - (Py_UNICODE *)PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3289 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3290 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3291 "ascii", "ordinal not in range(128)",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
3292 &starts, &e, &startinpos, &endinpos, &exc, &s,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3293 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3294 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3295 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3296 }
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
3297 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3298 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
3299 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3300 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3301 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3302 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3303
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3304 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3305 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3306 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3307 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3308 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3309 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3310
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3311 PyObject *PyUnicode_EncodeASCII(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3312 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3313 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3314 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3315 return unicode_encode_ucs1(p, size, errors, 128);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3316 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3317
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3318 PyObject *PyUnicode_AsASCIIString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3319 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3320 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3321 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3322 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3323 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3324 return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3325 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3326 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3327 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3328
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
3329 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
3330
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3331 /* --- MBCS codecs for Windows -------------------------------------------- */
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
3332
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3333 #if SIZEOF_INT < SIZEOF_SSIZE_T
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3334 #define NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3335 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3336
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3337 /* XXX This code is limited to "true" double-byte encodings, as
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3338 a) it assumes an incomplete character consists of a single byte, and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3339 b) IsDBCSLeadByte (probably) does not work for non-DBCS multi-byte
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3340 encodings, see IsDBCSLeadByteEx documentation. */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3341
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3342 static int is_dbcs_lead_byte(const char *s, int offset)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3343 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3344 const char *curr = s + offset;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3345
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3346 if (IsDBCSLeadByte(*curr)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3347 const char *prev = CharPrev(s, curr);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3348 return (prev == curr) || !IsDBCSLeadByte(*prev) || (curr - prev == 2);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3349 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3350 return 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3351 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3352
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3353 /*
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3354 * Decode MBCS string into unicode object. If 'final' is set, converts
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3355 * trailing lead-byte too. Returns consumed size if succeed, -1 otherwise.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3356 */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3357 static int decode_mbcs(PyUnicodeObject **v,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3358 const char *s, /* MBCS string */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3359 int size, /* sizeof MBCS string */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3360 int final)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3361 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3362 Py_UNICODE *p;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3363 Py_ssize_t n = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3364 int usize = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3365
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3366 assert(size >= 0);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3367
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3368 /* Skip trailing lead-byte unless 'final' is set */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3369 if (!final && size >= 1 && is_dbcs_lead_byte(s, size - 1))
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3370 --size;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3371
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3372 /* First get the size of the result */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3373 if (size > 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3374 usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3375 if (usize == 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3376 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3377 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3378 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3379 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3380
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3381 if (*v == NULL) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3382 /* Create unicode object */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3383 *v = _PyUnicode_New(usize);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3384 if (*v == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3385 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3386 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3387 else {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3388 /* Extend unicode object */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3389 n = PyUnicode_GET_SIZE(*v);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3390 if (_PyUnicode_Resize(v, n + usize) < 0)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3391 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3392 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3393
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3394 /* Do the conversion */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3395 if (size > 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3396 p = PyUnicode_AS_UNICODE(*v) + n;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3397 if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3398 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3399 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3400 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3401 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3402
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3403 return size;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3404 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3405
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3406 PyObject *PyUnicode_DecodeMBCSStateful(const char *s,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3407 Py_ssize_t size,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3408 const char *errors,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3409 Py_ssize_t *consumed)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3410 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3411 PyUnicodeObject *v = NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3412 int done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3413
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3414 if (consumed)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3415 *consumed = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3416
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3417 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3418 retry:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3419 if (size > INT_MAX)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3420 done = decode_mbcs(&v, s, INT_MAX, 0);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3421 else
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3422 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3423 done = decode_mbcs(&v, s, (int)size, !consumed);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3424
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3425 if (done < 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3426 Py_XDECREF(v);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3427 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3428 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3429
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3430 if (consumed)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3431 *consumed += done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3432
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3433 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3434 if (size > INT_MAX) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3435 s += done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3436 size -= done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3437 goto retry;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3438 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3439 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3440
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3441 return (PyObject *)v;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3442 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3443
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3444 PyObject *PyUnicode_DecodeMBCS(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3445 Py_ssize_t size,
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3446 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3447 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3448 return PyUnicode_DecodeMBCSStateful(s, size, errors, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3449 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3450
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3451 /*
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3452 * Convert unicode into string object (MBCS).
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3453 * Returns 0 if succeed, -1 otherwise.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3454 */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3455 static int encode_mbcs(PyObject **repr,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3456 const Py_UNICODE *p, /* unicode */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3457 int size) /* size of unicode */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3458 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3459 int mbcssize = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3460 Py_ssize_t n = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3461
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3462 assert(size >= 0);
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3463
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3464 /* First get the size of the result */
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3465 if (size > 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3466 mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3467 if (mbcssize == 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3468 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3469 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3470 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3471 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3472
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3473 if (*repr == NULL) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3474 /* Create string object */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3475 *repr = PyBytes_FromStringAndSize(NULL, mbcssize);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3476 if (*repr == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3477 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3478 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3479 else {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3480 /* Extend string object */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3481 n = PyBytes_Size(*repr);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3482 if (PyBytes_Resize(*repr, n + mbcssize) < 0)
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3483 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3484 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3485
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3486 /* Do the conversion */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3487 if (size > 0) {
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3488 char *s = PyBytes_AS_STRING(*repr) + n;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3489 if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3490 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3491 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3492 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3493 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3494
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3495 return 0;
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3496 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3497
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3498 PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3499 Py_ssize_t size,
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3500 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3501 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3502 PyObject *repr = NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3503 int ret;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3504
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3505 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3506 retry:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3507 if (size > INT_MAX)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3508 ret = encode_mbcs(&repr, p, INT_MAX);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3509 else
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3510 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3511 ret = encode_mbcs(&repr, p, (int)size);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3512
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3513 if (ret < 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3514 Py_XDECREF(repr);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3515 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3516 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3517
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3518 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3519 if (size > INT_MAX) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3520 p += INT_MAX;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3521 size -= INT_MAX;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3522 goto retry;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3523 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3524 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3525
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3526 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3527 }
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
3528
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3529 PyObject *PyUnicode_AsMBCSString(PyObject *unicode)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3530 {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3531 if (!PyUnicode_Check(unicode)) {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3532 PyErr_BadArgument();
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3533 return NULL;
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3534 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3535 return PyUnicode_EncodeMBCS(PyUnicode_AS_UNICODE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3536 PyUnicode_GET_SIZE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3537 NULL);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3538 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3539
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3540 #undef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3541
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
3542 #endif /* MS_WINDOWS */
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3543
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3544 /* --- Character Mapping Codec -------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3545
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3546 PyObject *PyUnicode_DecodeCharmap(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3547 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3548 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3549 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3550 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3551 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3552 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3553 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3554 Py_ssize_t outpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3555 const char *e;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3556 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3557 Py_UNICODE *p;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3558 Py_ssize_t extrachars = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3559 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3560 PyObject *exc = NULL;
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3561 Py_UNICODE *mapstring = NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3562 Py_ssize_t maplen = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3563
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3564 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3565 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3566 return PyUnicode_DecodeLatin1(s, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3567
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3568 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3569 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3570 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3571 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3572 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3573 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3574 e = s + size;
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3575 if (PyUnicode_CheckExact(mapping)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3576 mapstring = PyUnicode_AS_UNICODE(mapping);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3577 maplen = PyUnicode_GET_SIZE(mapping);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3578 while (s < e) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3579 unsigned char ch = *s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3580 Py_UNICODE x = 0xfffe; /* illegal value */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3581
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3582 if (ch < maplen)
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3583 x = mapstring[ch];
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3584
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3585 if (x == 0xfffe) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3586 /* undefined mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3587 outpos = p-PyUnicode_AS_UNICODE(v);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3588 startinpos = s-starts;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3589 endinpos = startinpos+1;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3590 if (unicode_decode_call_errorhandler(
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3591 errors, &errorHandler,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3592 "charmap", "character maps to <undefined>",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
3593 &starts, &e, &startinpos, &endinpos, &exc, &s,
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3594 (PyObject **)&v, &outpos, &p)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3595 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3596 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3597 continue;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3598 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3599 *p++ = x;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3600 ++s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3601 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3602 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3603 else {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3604 while (s < e) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3605 unsigned char ch = *s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3606 PyObject *w, *x;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3607
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3608 /* Get mapping (char ordinal -> integer, Unicode char or None) */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3609 w = PyInt_FromLong((long)ch);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3610 if (w == NULL)
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3611 goto onError;
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3612 x = PyObject_GetItem(mapping, w);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3613 Py_DECREF(w);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3614 if (x == NULL) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3615 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3616 /* No mapping found means: mapping is undefined. */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3617 PyErr_Clear();
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3618 x = Py_None;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3619 Py_INCREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3620 } else
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3621 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3622 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3623
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3624 /* Apply mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3625 if (PyInt_Check(x)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3626 long value = PyInt_AS_LONG(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3627 if (value < 0 || value > 65535) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3628 PyErr_SetString(PyExc_TypeError,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3629 "character mapping must be in range(65536)");
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3630 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3631 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3632 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3633 *p++ = (Py_UNICODE)value;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3634 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3635 else if (x == Py_None) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3636 /* undefined mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3637 outpos = p-PyUnicode_AS_UNICODE(v);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3638 startinpos = s-starts;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3639 endinpos = startinpos+1;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3640 if (unicode_decode_call_errorhandler(
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3641 errors, &errorHandler,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3642 "charmap", "character maps to <undefined>",
42408
504084c04ac0 Bytes (which are the input for decoding) are mutable now. If a decoding
Walter Dörwald <walter@livinglogic.de>
parents: 42335
diff changeset
3643 &starts, &e, &startinpos, &endinpos, &exc, &s,
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3644 (PyObject **)&v, &outpos, &p)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3645 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3646 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3647 }
35962
73ebdc62bf8f Fix leaked reference to None.
Walter Dörwald <walter@livinglogic.de>
parents: 35827
diff changeset
3648 Py_DECREF(x);
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3649 continue;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3650 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3651 else if (PyUnicode_Check(x)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3652 Py_ssize_t targetsize = PyUnicode_GET_SIZE(x);
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3653
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3654 if (targetsize == 1)
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3655 /* 1-1 mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3656 *p++ = *PyUnicode_AS_UNICODE(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3657
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3658 else if (targetsize > 1) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3659 /* 1-n mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3660 if (targetsize > extrachars) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3661 /* resize first */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3662 Py_ssize_t oldpos = p - PyUnicode_AS_UNICODE(v);
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3663 Py_ssize_t needed = (targetsize - extrachars) + \
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3664 (targetsize << 2);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3665 extrachars += needed;
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
3666 /* XXX overflow detection missing */
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3667 if (_PyUnicode_Resize(&v,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3668 PyUnicode_GET_SIZE(v) + needed) < 0) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3669 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3670 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3671 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3672 p = PyUnicode_AS_UNICODE(v) + oldpos;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3673 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3674 Py_UNICODE_COPY(p,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3675 PyUnicode_AS_UNICODE(x),
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3676 targetsize);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3677 p += targetsize;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3678 extrachars -= targetsize;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3679 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3680 /* 1-0 mapping: skip the character */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3681 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3682 else {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3683 /* wrong return value */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3684 PyErr_SetString(PyExc_TypeError,
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3685 "character mapping must return integer, None or unicode");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3686 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3687 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3688 }
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3689 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3690 ++s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3691 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3692 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3693 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3694 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3695 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3696 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3697 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3698 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3699
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3700 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3701 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3702 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3703 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3704 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3705 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3706
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3707 /* Charmap encoding: the lookup table */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3708
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3709 struct encoding_map{
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3710 PyObject_HEAD
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3711 unsigned char level1[32];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3712 int count2, count3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3713 unsigned char level23[1];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3714 };
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3715
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3716 static PyObject*
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3717 encoding_map_size(PyObject *obj, PyObject* args)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3718 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3719 struct encoding_map *map = (struct encoding_map*)obj;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3720 return PyInt_FromLong(sizeof(*map) - 1 + 16*map->count2 +
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3721 128*map->count3);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3722 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3723
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3724 static PyMethodDef encoding_map_methods[] = {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3725 {"size", encoding_map_size, METH_NOARGS,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3726 PyDoc_STR("Return the size (in bytes) of this object") },
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3727 { 0 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3728 };
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3729
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3730 static void
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3731 encoding_map_dealloc(PyObject* o)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3732 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3733 PyObject_FREE(o);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3734 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3735
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3736 static PyTypeObject EncodingMapType = {
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
3737 PyVarObject_HEAD_INIT(NULL, 0)
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3738 "EncodingMap", /*tp_name*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3739 sizeof(struct encoding_map), /*tp_basicsize*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3740 0, /*tp_itemsize*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3741 /* methods */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3742 encoding_map_dealloc, /*tp_dealloc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3743 0, /*tp_print*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3744 0, /*tp_getattr*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3745 0, /*tp_setattr*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3746 0, /*tp_compare*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3747 0, /*tp_repr*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3748 0, /*tp_as_number*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3749 0, /*tp_as_sequence*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3750 0, /*tp_as_mapping*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3751 0, /*tp_hash*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3752 0, /*tp_call*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3753 0, /*tp_str*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3754 0, /*tp_getattro*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3755 0, /*tp_setattro*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3756 0, /*tp_as_buffer*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3757 Py_TPFLAGS_DEFAULT, /*tp_flags*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3758 0, /*tp_doc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3759 0, /*tp_traverse*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3760 0, /*tp_clear*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3761 0, /*tp_richcompare*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3762 0, /*tp_weaklistoffset*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3763 0, /*tp_iter*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3764 0, /*tp_iternext*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3765 encoding_map_methods, /*tp_methods*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3766 0, /*tp_members*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3767 0, /*tp_getset*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3768 0, /*tp_base*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3769 0, /*tp_dict*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3770 0, /*tp_descr_get*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3771 0, /*tp_descr_set*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3772 0, /*tp_dictoffset*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3773 0, /*tp_init*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3774 0, /*tp_alloc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3775 0, /*tp_new*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3776 0, /*tp_free*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3777 0, /*tp_is_gc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3778 };
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3779
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3780 PyObject*
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3781 PyUnicode_BuildEncodingMap(PyObject* string)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3782 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3783 Py_UNICODE *decode;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3784 PyObject *result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3785 struct encoding_map *mresult;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3786 int i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3787 int need_dict = 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3788 unsigned char level1[32];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3789 unsigned char level2[512];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3790 unsigned char *mlevel1, *mlevel2, *mlevel3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3791 int count2 = 0, count3 = 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3792
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3793 if (!PyUnicode_Check(string) || PyUnicode_GetSize(string) != 256) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3794 PyErr_BadArgument();
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3795 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3796 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3797 decode = PyUnicode_AS_UNICODE(string);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3798 memset(level1, 0xFF, sizeof level1);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3799 memset(level2, 0xFF, sizeof level2);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3800
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3801 /* If there isn't a one-to-one mapping of NULL to \0,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3802 or if there are non-BMP characters, we need to use
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3803 a mapping dictionary. */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3804 if (decode[0] != 0)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3805 need_dict = 1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3806 for (i = 1; i < 256; i++) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3807 int l1, l2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3808 if (decode[i] == 0
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3809 #ifdef Py_UNICODE_WIDE
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3810 || decode[i] > 0xFFFF
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3811 #endif
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3812 ) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3813 need_dict = 1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3814 break;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3815 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3816 if (decode[i] == 0xFFFE)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3817 /* unmapped character */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3818 continue;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3819 l1 = decode[i] >> 11;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3820 l2 = decode[i] >> 7;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3821 if (level1[l1] == 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3822 level1[l1] = count2++;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3823 if (level2[l2] == 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3824 level2[l2] = count3++;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3825 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3826
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3827 if (count2 >= 0xFF || count3 >= 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3828 need_dict = 1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3829
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3830 if (need_dict) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3831 PyObject *result = PyDict_New();
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3832 PyObject *key, *value;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3833 if (!result)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3834 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3835 for (i = 0; i < 256; i++) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3836 key = value = NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3837 key = PyInt_FromLong(decode[i]);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3838 value = PyInt_FromLong(i);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3839 if (!key || !value)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3840 goto failed1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3841 if (PyDict_SetItem(result, key, value) == -1)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3842 goto failed1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3843 Py_DECREF(key);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3844 Py_DECREF(value);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3845 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3846 return result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3847 failed1:
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3848 Py_XDECREF(key);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3849 Py_XDECREF(value);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3850 Py_DECREF(result);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3851 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3852 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3853
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3854 /* Create a three-level trie */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3855 result = PyObject_MALLOC(sizeof(struct encoding_map) +
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3856 16*count2 + 128*count3 - 1);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3857 if (!result)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3858 return PyErr_NoMemory();
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3859 PyObject_Init(result, &EncodingMapType);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3860 mresult = (struct encoding_map*)result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3861 mresult->count2 = count2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3862 mresult->count3 = count3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3863 mlevel1 = mresult->level1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3864 mlevel2 = mresult->level23;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3865 mlevel3 = mresult->level23 + 16*count2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3866 memcpy(mlevel1, level1, 32);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3867 memset(mlevel2, 0xFF, 16*count2);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3868 memset(mlevel3, 0, 128*count3);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3869 count3 = 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3870 for (i = 1; i < 256; i++) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3871 int o1, o2, o3, i2, i3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3872 if (decode[i] == 0xFFFE)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3873 /* unmapped character */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3874 continue;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3875 o1 = decode[i]>>11;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3876 o2 = (decode[i]>>7) & 0xF;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3877 i2 = 16*mlevel1[o1] + o2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3878 if (mlevel2[i2] == 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3879 mlevel2[i2] = count3++;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3880 o3 = decode[i] & 0x7F;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3881 i3 = 128*mlevel2[i2] + o3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3882 mlevel3[i3] = i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3883 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3884 return result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3885 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3886
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3887 static int
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3888 encoding_map_lookup(Py_UNICODE c, PyObject *mapping)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3889 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3890 struct encoding_map *map = (struct encoding_map*)mapping;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3891 int l1 = c>>11;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3892 int l2 = (c>>7) & 0xF;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3893 int l3 = c & 0x7F;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3894 int i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3895
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3896 #ifdef Py_UNICODE_WIDE
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3897 if (c > 0xFFFF) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3898 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3899 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3900 #endif
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3901 if (c == 0)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3902 return 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3903 /* level 1*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3904 i = map->level1[l1];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3905 if (i == 0xFF) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3906 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3907 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3908 /* level 2*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3909 i = map->level23[16*i+l2];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3910 if (i == 0xFF) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3911 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3912 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3913 /* level 3 */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3914 i = map->level23[16*map->count2 + 128*i + l3];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3915 if (i == 0) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3916 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3917 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3918 return i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3919 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3920
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3921 /* Lookup the character ch in the mapping. If the character
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3922 can't be found, Py_None is returned (or NULL, if another
35806
44fe157255bc fix typos, mostly in comments
Fred Drake <fdrake@acm.org>
parents: 35758
diff changeset
3923 error occurred). */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3924 static PyObject *charmapencode_lookup(Py_UNICODE c, PyObject *mapping)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3925 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3926 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3927 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3928
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3929 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3930 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3931 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3932 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3933 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3934 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3935 /* No mapping found means: mapping is undefined. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3936 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3937 x = Py_None;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3938 Py_INCREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3939 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3940 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3941 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3942 }
27175
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping
Walter Dörwald <walter@livinglogic.de>
parents: 27170
diff changeset
3943 else if (x == Py_None)
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping
Walter Dörwald <walter@livinglogic.de>
parents: 27170
diff changeset
3944 return x;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3945 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3946 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3947 if (value < 0 || value > 255) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3948 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3949 "character mapping must be in range(256)");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3950 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3951 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3952 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3953 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3954 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3955 else if (PyString_Check(x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3956 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3957 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3958 /* wrong return value */
41658
7fe6afcbd3f2 Mention type in the exception message.
Walter Dörwald <walter@livinglogic.de>
parents: 41629
diff changeset
3959 PyErr_Format(PyExc_TypeError,
7fe6afcbd3f2 Mention type in the exception message.
Walter Dörwald <walter@livinglogic.de>
parents: 41629
diff changeset
3960 "character mapping must return integer, None or str8, not %.400s",
7fe6afcbd3f2 Mention type in the exception message.
Walter Dörwald <walter@livinglogic.de>
parents: 41629
diff changeset
3961 x->ob_type->tp_name);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3962 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3963 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3964 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3965 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3966
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3967 static int
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3968 charmapencode_resize(PyObject *outobj, Py_ssize_t *outpos, Py_ssize_t requiredsize)
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3969 {
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3970 Py_ssize_t outsize = PyBytes_GET_SIZE( outobj);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3971 /* exponentially overallocate to minimize reallocations */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3972 if (requiredsize < 2*outsize)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3973 requiredsize = 2*outsize;
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3974 if (PyBytes_Resize(outobj, requiredsize)) {
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3975 Py_DECREF(outobj);
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3976 return -1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3977 }
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3978 return 0;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3979 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3980
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3981 typedef enum charmapencode_result {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3982 enc_SUCCESS, enc_FAILED, enc_EXCEPTION
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3983 }charmapencode_result;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3984 /* lookup the character, put the result in the output string and adjust
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3985 various state variables. Resize the output bytes object if not enough
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3986 space is available. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3987 was put in the output buffer, or Py_None, if the mapping was undefined
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3988 (in which case no character was written) or NULL, if a
35827
ab3d1e6a0237 Another comment typo fix
Andrew M. Kuchling <amk@amk.ca>
parents: 35826
diff changeset
3989 reallocation error occurred. The caller must decref the result */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3990 static
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3991 charmapencode_result charmapencode_output(Py_UNICODE c, PyObject *mapping,
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3992 PyObject *outobj, Py_ssize_t *outpos)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3993 {
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3994 PyObject *rep;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3995 char *outstart;
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3996 Py_ssize_t outsize = PyBytes_GET_SIZE(outobj);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3997
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
3998 if (Py_Type(mapping) == &EncodingMapType) {
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3999 int res = encoding_map_lookup(c, mapping);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4000 Py_ssize_t requiredsize = *outpos+1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4001 if (res == -1)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4002 return enc_FAILED;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4003 if (outsize<requiredsize)
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4004 if (charmapencode_resize(outobj, outpos, requiredsize))
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4005 return enc_EXCEPTION;
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4006 outstart = PyBytes_AS_STRING(outobj);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4007 outstart[(*outpos)++] = (char)res;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4008 return enc_SUCCESS;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4009 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4010
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4011 rep = charmapencode_lookup(c, mapping);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4012 if (rep==NULL)
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4013 return enc_EXCEPTION;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4014 else if (rep==Py_None) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4015 Py_DECREF(rep);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4016 return enc_FAILED;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4017 } else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4018 if (PyInt_Check(rep)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4019 Py_ssize_t requiredsize = *outpos+1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4020 if (outsize<requiredsize)
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4021 if (charmapencode_resize(outobj, outpos, requiredsize)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4022 Py_DECREF(rep);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4023 return enc_EXCEPTION;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4024 }
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4025 outstart = PyBytes_AS_STRING(outobj);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4026 outstart[(*outpos)++] = (char)PyInt_AS_LONG(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4027 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4028 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4029 const char *repchars = PyString_AS_STRING(rep);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4030 Py_ssize_t repsize = PyString_GET_SIZE(rep);
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4031 Py_ssize_t requiredsize = *outpos+repsize;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4032 if (outsize<requiredsize)
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4033 if (charmapencode_resize(outobj, outpos, requiredsize)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4034 Py_DECREF(rep);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4035 return enc_EXCEPTION;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4036 }
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4037 outstart = PyBytes_AS_STRING(outobj);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4038 memcpy(outstart + *outpos, repchars, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4039 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4040 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4041 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4042 Py_DECREF(rep);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4043 return enc_SUCCESS;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4044 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4045
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4046 /* handle an error in PyUnicode_EncodeCharmap
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4047 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4048 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4049 int charmap_encoding_error(
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4050 const Py_UNICODE *p, Py_ssize_t size, Py_ssize_t *inpos, PyObject *mapping,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4051 PyObject **exceptionObject,
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
4052 int *known_errorHandler, PyObject **errorHandler, const char *errors,
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4053 PyObject *res, Py_ssize_t *respos)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4054 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4055 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4056 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4057 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4058 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4059 /* startpos for collecting unencodable chars */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4060 Py_ssize_t collstartpos = *inpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4061 Py_ssize_t collendpos = *inpos+1;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4062 Py_ssize_t collpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4063 char *encoding = "charmap";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4064 char *reason = "character maps to <undefined>";
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4065 charmapencode_result x;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4066
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4067 /* find all unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4068 while (collendpos < size) {
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4069 PyObject *rep;
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
4070 if (Py_Type(mapping) == &EncodingMapType) {
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4071 int res = encoding_map_lookup(p[collendpos], mapping);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4072 if (res != -1)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4073 break;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4074 ++collendpos;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4075 continue;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4076 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4077
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4078 rep = charmapencode_lookup(p[collendpos], mapping);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4079 if (rep==NULL)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4080 return -1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4081 else if (rep!=Py_None) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4082 Py_DECREF(rep);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4083 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4084 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4085 Py_DECREF(rep);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4086 ++collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4087 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4088 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4089 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4090 if (*known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4091 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4092 *known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4093 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4094 *known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4095 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4096 *known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4097 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4098 *known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4099 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4100 *known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4101 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4102 switch (*known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4103 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4104 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4105 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4106 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4107 for (collpos = collstartpos; collpos<collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4108 x = charmapencode_output('?', mapping, res, respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4109 if (x==enc_EXCEPTION) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4110 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4111 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4112 else if (x==enc_FAILED) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4113 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4114 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4115 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4116 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4117 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4118 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4119 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4120 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4121 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4122 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4123 for (collpos = collstartpos; collpos < collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4124 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4125 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4126 sprintf(buffer, "&#%d;", (int)p[collpos]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4127 for (cp = buffer; *cp; ++cp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4128 x = charmapencode_output(*cp, mapping, res, respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4129 if (x==enc_EXCEPTION)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4130 return -1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4131 else if (x==enc_FAILED) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4132 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4133 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4134 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4135 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4136 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4137 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4138 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4139 default:
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
4140 repunicode = unicode_encode_call_errorhandler(errors, errorHandler,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4141 encoding, reason, p, size, exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4142 collstartpos, collendpos, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4143 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4144 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4145 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4146 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4147 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4148 x = charmapencode_output(*uni2, mapping, res, respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4149 if (x==enc_EXCEPTION) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4150 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4151 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4152 else if (x==enc_FAILED) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4153 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4154 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4155 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4156 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4157 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4158 *inpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4159 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4160 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4161 return 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4162 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4163
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4164 PyObject *PyUnicode_EncodeCharmap(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4165 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4166 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4167 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4168 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4169 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4170 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4171 /* current input position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4172 Py_ssize_t inpos = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4173 /* current output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4174 Py_ssize_t respos = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4175 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4176 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4177 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4178 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4179 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4180 int known_errorHandler = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4181
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4182 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4183 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4184 return PyUnicode_EncodeLatin1(p, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4185
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4186 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4187 replacements, if we need more, we'll resize */
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4188 res = PyBytes_FromStringAndSize(NULL, size);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4189 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4190 goto onError;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
4191 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4192 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4193
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4194 while (inpos<size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4195 /* try to encode it */
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4196 charmapencode_result x = charmapencode_output(p[inpos], mapping, res, &respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4197 if (x==enc_EXCEPTION) /* error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4198 goto onError;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4199 if (x==enc_FAILED) { /* unencodable character */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4200 if (charmap_encoding_error(p, size, &inpos, mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4201 &exc,
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
4202 &known_errorHandler, &errorHandler, errors,
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4203 res, &respos)) {
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
4204 goto onError;
30162
443533d08783 Fix another refcounting leak in PyUnicode_EncodeCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30160
diff changeset
4205 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4206 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4207 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4208 /* done with this character => adjust input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4209 ++inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4210 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4211
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4212 /* Resize if we allocated to much */
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4213 if (respos<PyBytes_GET_SIZE(res)) {
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4214 if (PyBytes_Resize(res, respos))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4215 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4216 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4217 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4218 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4219 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4220
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4221 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4222 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4223 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4224 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4225 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4226 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4227
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4228 PyObject *PyUnicode_AsCharmapString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4229 PyObject *mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4230 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4231 if (!PyUnicode_Check(unicode) || mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4232 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4233 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4234 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4235 return PyUnicode_EncodeCharmap(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4236 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4237 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4238 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4239 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4240
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4241 /* create or adjust a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4242 static void make_translate_exception(PyObject **exceptionObject,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4243 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4244 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4245 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4246 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4247 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4248 *exceptionObject = PyUnicodeTranslateError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4249 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4250 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4251 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4252 if (PyUnicodeTranslateError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4253 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4254 if (PyUnicodeTranslateError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4255 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4256 if (PyUnicodeTranslateError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4257 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4258 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4259 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4260 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4261 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4262 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4263 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4264
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4265 /* raises a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4266 static void raise_translate_exception(PyObject **exceptionObject,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4267 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4268 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4269 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4270 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4271 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4272 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4273 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4274 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4275 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4276
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4277 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4278 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4279 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4280 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4281 static PyObject *unicode_translate_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4282 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4283 const char *reason,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4284 const Py_UNICODE *unicode, Py_ssize_t size, PyObject **exceptionObject,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4285 Py_ssize_t startpos, Py_ssize_t endpos,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4286 Py_ssize_t *newpos)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4287 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
4288 static char *argparse = "O!n;translating error handler must return (unicode, int) tuple";
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
4289
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
4290 Py_ssize_t i_newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4291 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4292 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4293
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4294 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4295 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4296 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4297 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4298 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4299
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4300 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4301 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4302 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4303 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4304
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4305 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4306 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4307 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4308 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4309 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4310 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4311 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4312 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4313 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4314 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4315 &resunicode, &i_newpos)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4316 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4317 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4318 }
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4319 if (i_newpos<0)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4320 *newpos = size+i_newpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4321 else
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4322 *newpos = i_newpos;
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4323 if (*newpos<0 || *newpos>size) {
36510
0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat.
Martin v. Löwis <martin@v.loewis.de>
parents: 36508
diff changeset
4324 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", *newpos);
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4325 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4326 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4327 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4328 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4329 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4330 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4331 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4332
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4333 /* Lookup the character ch in the mapping and put the result in result,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4334 which must be decrefed by the caller.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4335 Return 0 on success, -1 on error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4336 static
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4337 int charmaptranslate_lookup(Py_UNICODE c, PyObject *mapping, PyObject **result)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4338 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4339 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4340 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4341
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4342 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4343 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4344 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4345 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4346 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4347 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4348 /* No mapping found means: use 1:1 mapping. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4349 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4350 *result = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4351 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4352 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4353 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4354 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4355 else if (x == Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4356 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4357 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4358 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4359 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4360 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4361 long max = PyUnicode_GetMax();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4362 if (value < 0 || value > max) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4363 PyErr_Format(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4364 "character mapping must be in range(0x%lx)", max+1);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4365 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4366 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4367 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4368 *result = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4369 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4370 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4371 else if (PyUnicode_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4372 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4373 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4374 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4375 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4376 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4377 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4378 "character mapping must return integer, None or unicode");
30164
7cb4ad5932e6 Fix refcounting leak in charmaptranslate_lookup()
Walter Dörwald <walter@livinglogic.de>
parents: 30162
diff changeset
4379 Py_DECREF(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4380 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4381 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4382 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4383 /* ensure that *outobj is at least requiredsize characters long,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4384 if not reallocate and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4385 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4386 static
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4387 int charmaptranslate_makespace(PyObject **outobj, Py_UNICODE **outp,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4388 Py_ssize_t requiredsize)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4389 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4390 Py_ssize_t oldsize = PyUnicode_GET_SIZE(*outobj);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4391 if (requiredsize > oldsize) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4392 /* remember old output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4393 Py_ssize_t outpos = *outp-PyUnicode_AS_UNICODE(*outobj);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4394 /* exponentially overallocate to minimize reallocations */
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4395 if (requiredsize < 2 * oldsize)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4396 requiredsize = 2 * oldsize;
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
4397 if (_PyUnicode_Resize(outobj, requiredsize) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4398 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4399 *outp = PyUnicode_AS_UNICODE(*outobj) + outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4400 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4401 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4402 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4403 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4404 various state variables. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4405 was put in the output buffer in *result, or Py_None, if the mapping was
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4406 undefined (in which case no character was written).
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4407 The called must decref result.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4408 Return 0 on success, -1 on error. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4409 static
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4410 int charmaptranslate_output(const Py_UNICODE *startinp, const Py_UNICODE *curinp,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4411 Py_ssize_t insize, PyObject *mapping, PyObject **outobj, Py_UNICODE **outp,
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4412 PyObject **res)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4413 {
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4414 if (charmaptranslate_lookup(*curinp, mapping, res))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4415 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4416 if (*res==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4417 /* not found => default to 1:1 mapping */
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4418 *(*outp)++ = *curinp;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4419 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4420 else if (*res==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4421 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4422 else if (PyInt_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4423 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4424 *(*outp)++ = (Py_UNICODE)PyInt_AS_LONG(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4425 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4426 else if (PyUnicode_Check(*res)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4427 Py_ssize_t repsize = PyUnicode_GET_SIZE(*res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4428 if (repsize==1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4429 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4430 *(*outp)++ = *PyUnicode_AS_UNICODE(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4431 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4432 else if (repsize!=0) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4433 /* more than one character */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4434 Py_ssize_t requiredsize = (*outp-PyUnicode_AS_UNICODE(*outobj)) +
31455
37d501b9e2d5 Fix reallocation bug in unicode.translate(): The code was comparing
Walter Dörwald <walter@livinglogic.de>
parents: 31325
diff changeset
4435 (insize - (curinp-startinp)) +
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4436 repsize - 1;
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4437 if (charmaptranslate_makespace(outobj, outp, requiredsize))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4438 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4439 memcpy(*outp, PyUnicode_AS_UNICODE(*res), sizeof(Py_UNICODE)*repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4440 *outp += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4441 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4442 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4443 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4444 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4445 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4446 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4447
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4448 PyObject *PyUnicode_TranslateCharmap(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4449 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4450 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4451 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4452 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4453 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4454 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4455 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4456 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4457 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4458 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4459 Py_UNICODE *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4460 /* current output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4461 Py_ssize_t respos = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4462 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4463 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4464 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4465 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4466 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4467 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4468 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4469
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4470 if (mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4471 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4472 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4473 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4474
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4475 /* allocate enough for a simple 1:1 translation without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4476 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4477 res = PyUnicode_FromUnicode(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4478 if (res == NULL)
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4479 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4480 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4481 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4482 str = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4483
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4484 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4485 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4486 PyObject *x = NULL;
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4487 if (charmaptranslate_output(startp, p, size, mapping, &res, &str, &x)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4488 Py_XDECREF(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4489 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4490 }
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
4491 Py_XDECREF(x);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4492 if (x!=Py_None) /* it worked => adjust input pointer */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4493 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4494 else { /* untranslatable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4495 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4496 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4497 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4498 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4499 /* startpos for collecting untranslatable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4500 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4501 const Py_UNICODE *collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4502 const Py_UNICODE *coll;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4503
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4504 /* find all untranslatable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4505 while (collend < endp) {
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4506 if (charmaptranslate_lookup(*collend, mapping, &x))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4507 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4508 Py_XDECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4509 if (x!=Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4510 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4511 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4512 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4513 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4514 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4515 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4516 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4517 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4518 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4519 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4520 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4521 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4522 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4523 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4524 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4525 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4526 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4527 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4528 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4529 raise_translate_exception(&exc, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4530 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4531 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4532 /* No need to check for space, this is a 1:1 replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4533 for (coll = collstart; coll<collend; ++coll)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4534 *str++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4535 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4536 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4537 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4538 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4539 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4540 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4541 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4542 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4543 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4544 sprintf(buffer, "&#%d;", (int)*p);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4545 if (charmaptranslate_makespace(&res, &str,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4546 (str-PyUnicode_AS_UNICODE(res))+strlen(buffer)+(endp-collend)))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4547 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4548 for (cp = buffer; *cp; ++cp)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4549 *str++ = *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4550 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4551 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4552 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4553 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4554 repunicode = unicode_translate_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4555 reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4556 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4557 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4558 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4559 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4560 repsize = PyUnicode_GET_SIZE(repunicode);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4561 if (charmaptranslate_makespace(&res, &str,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4562 (str-PyUnicode_AS_UNICODE(res))+repsize+(endp-collend))) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4563 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4564 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4565 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4566 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4567 *str++ = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4568 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4569 Py_DECREF(repunicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4570 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4571 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4572 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4573 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4574 respos = str-PyUnicode_AS_UNICODE(res);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4575 if (respos<PyUnicode_GET_SIZE(res)) {
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
4576 if (_PyUnicode_Resize(&res, respos) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4577 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4578 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4579 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4580 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4581 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4582
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4583 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4584 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4585 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4586 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4587 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4588 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4589
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4590 PyObject *PyUnicode_Translate(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4591 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4592 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4593 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4594 PyObject *result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4595
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4596 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4597 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4598 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4599 result = PyUnicode_TranslateCharmap(PyUnicode_AS_UNICODE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4600 PyUnicode_GET_SIZE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4601 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4602 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4603 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4604 return result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4605
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4606 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4607 Py_XDECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4608 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4609 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4610
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4611 /* --- Decimal Encoder ---------------------------------------------------- */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4612
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4613 int PyUnicode_EncodeDecimal(Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4614 Py_ssize_t length,
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4615 char *output,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4616 const char *errors)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4617 {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4618 Py_UNICODE *p, *end;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4619 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4620 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4621 const char *encoding = "decimal";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4622 const char *reason = "invalid decimal Unicode string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4623 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4624 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4625 int known_errorHandler = -1;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4626
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4627 if (output == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4628 PyErr_BadArgument();
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4629 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4630 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4631
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4632 p = s;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4633 end = s + length;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4634 while (p < end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4635 register Py_UNICODE ch = *p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4636 int decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4637 PyObject *repunicode;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4638 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4639 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4640 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4641 Py_UNICODE *collstart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4642 Py_UNICODE *collend;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4643
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4644 if (Py_UNICODE_ISSPACE(ch)) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4645 *output++ = ' ';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4646 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4647 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4648 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4649 decimal = Py_UNICODE_TODECIMAL(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4650 if (decimal >= 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4651 *output++ = '0' + decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4652 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4653 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4654 }
12275
cbfa95a6a03e Conrad Huang points out that "if (0 < ch < 256)", while legal C,
Guido van Rossum <guido@python.org>
parents: 12249
diff changeset
4655 if (0 < ch && ch < 256) {
12577
85f00f8cd674 Fix warning detected by VC++ on assignment of Py_UNICODE to char.
Guido van Rossum <guido@python.org>
parents: 12576
diff changeset
4656 *output++ = (char)ch;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4657 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4658 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4659 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4660 /* All other characters are considered unencodable */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4661 collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4662 collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4663 while (collend < end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4664 if ((0 < *collend && *collend < 256) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4665 !Py_UNICODE_ISSPACE(*collend) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4666 Py_UNICODE_TODECIMAL(*collend))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4667 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4668 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4669 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4670 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4671 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4672 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4673 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4674 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4675 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4676 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4677 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4678 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4679 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4680 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4681 known_errorHandler = 0;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4682 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4683 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4684 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4685 raise_encode_exception(&exc, encoding, s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4686 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4687 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4688 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4689 *output++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4690 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4691 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4692 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4693 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4694 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4695 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4696 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4697 output += sprintf(output, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4698 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4699 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4700 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4701 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4702 encoding, reason, s, length, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4703 collstart-s, collend-s, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4704 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4705 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4706 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4707 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4708 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4709 Py_UNICODE ch = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4710 if (Py_UNICODE_ISSPACE(ch))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4711 *output++ = ' ';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4712 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4713 decimal = Py_UNICODE_TODECIMAL(ch);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4714 if (decimal >= 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4715 *output++ = '0' + decimal;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4716 else if (0 < ch && ch < 256)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4717 *output++ = (char)ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4718 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4719 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4720 raise_encode_exception(&exc, encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4721 s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4722 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4723 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4724 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4725 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4726 p = s + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4727 Py_DECREF(repunicode);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4728 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4729 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4730 /* 0-terminate the output string */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4731 *output++ = '\0';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4732 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4733 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4734 return 0;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4735
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4736 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4737 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4738 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4739 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4740 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4741
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4742 /* --- Helpers ------------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4743
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4744 #define STRINGLIB_CHAR Py_UNICODE
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4745
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4746 #define STRINGLIB_LEN PyUnicode_GET_SIZE
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4747 #define STRINGLIB_NEW PyUnicode_FromUnicode
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4748 #define STRINGLIB_STR PyUnicode_AS_UNICODE
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4749
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4750 Py_LOCAL_INLINE(int)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4751 STRINGLIB_CMP(const Py_UNICODE* str, const Py_UNICODE* other, Py_ssize_t len)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4752 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4753 if (str[0] != other[0])
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4754 return 1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4755 return memcmp((void*) str, (void*) other, len * sizeof(Py_UNICODE));
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4756 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4757
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4758 #define STRINGLIB_EMPTY unicode_empty
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4759
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4760 #include "stringlib/fastsearch.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4761
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4762 #include "stringlib/count.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4763 #include "stringlib/find.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4764 #include "stringlib/partition.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4765
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4766 /* helper macro to fixup start/end slice values */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4767 #define FIX_START_END(obj) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4768 if (start < 0) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4769 start += (obj)->length; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4770 if (start < 0) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4771 start = 0; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4772 if (end > (obj)->length) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4773 end = (obj)->length; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4774 if (end < 0) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4775 end += (obj)->length; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4776 if (end < 0) \
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
4777 end = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
4778
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4779 Py_ssize_t PyUnicode_Count(PyObject *str,
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4780 PyObject *substr,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4781 Py_ssize_t start,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4782 Py_ssize_t end)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4783 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4784 Py_ssize_t result;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4785 PyUnicodeObject* str_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4786 PyUnicodeObject* sub_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4787
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4788 str_obj = (PyUnicodeObject*) PyUnicode_FromObject(str);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4789 if (!str_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4790 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4791 sub_obj = (PyUnicodeObject*) PyUnicode_FromObject(substr);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4792 if (!sub_obj) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4793 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4794 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4795 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4796
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4797 FIX_START_END(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4798
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4799 result = stringlib_count(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4800 str_obj->str + start, end - start, sub_obj->str, sub_obj->length
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4801 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4802
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4803 Py_DECREF(sub_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4804 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4805
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4806 return result;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4807 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4808
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4809 Py_ssize_t PyUnicode_Find(PyObject *str,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4810 PyObject *sub,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4811 Py_ssize_t start,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4812 Py_ssize_t end,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4813 int direction)
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4814 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4815 Py_ssize_t result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4816
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4817 str = PyUnicode_FromObject(str);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4818 if (!str)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4819 return -2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4820 sub = PyUnicode_FromObject(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4821 if (!sub) {
12957
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
4822 Py_DECREF(str);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4823 return -2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4824 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4825
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4826 if (direction > 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4827 result = stringlib_find_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4828 PyUnicode_AS_UNICODE(str), PyUnicode_GET_SIZE(str),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4829 PyUnicode_AS_UNICODE(sub), PyUnicode_GET_SIZE(sub),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4830 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4831 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4832 else
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4833 result = stringlib_rfind_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4834 PyUnicode_AS_UNICODE(str), PyUnicode_GET_SIZE(str),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4835 PyUnicode_AS_UNICODE(sub), PyUnicode_GET_SIZE(sub),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4836 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4837 );
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4838
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4839 Py_DECREF(str);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4840 Py_DECREF(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4841
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4842 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4843 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4844
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4845 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4846 int tailmatch(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4847 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4848 Py_ssize_t start,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4849 Py_ssize_t end,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4850 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4851 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4852 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4853 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4854
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4855 FIX_START_END(self);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4856
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4857 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4858 if (end < start)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4859 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4860
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4861 if (direction > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4862 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4863 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4864 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4865 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4866 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4867 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4868
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4869 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4870 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4871
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4872 Py_ssize_t PyUnicode_Tailmatch(PyObject *str,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4873 PyObject *substr,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4874 Py_ssize_t start,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4875 Py_ssize_t end,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4876 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4877 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4878 Py_ssize_t result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4879
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4880 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4881 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4882 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4883 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4884 if (substr == NULL) {
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
4885 Py_DECREF(str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4886 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4887 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4888
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4889 result = tailmatch((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4890 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4891 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4892 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4893 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4894 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4895 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4896
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4897 /* Apply fixfct filter to the Unicode object self and return a
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4898 reference to the modified object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4899
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4900 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4901 PyObject *fixup(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4902 int (*fixfct)(PyUnicodeObject *s))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4903 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4904
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4905 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4906
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4907 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4908 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4909 return NULL;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4910
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4911 Py_UNICODE_COPY(u->str, self->str, self->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4912
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4913 if (!fixfct(u) && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4914 /* fixfct should return TRUE if it modified the buffer. If
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4915 FALSE, return a reference to the original buffer instead
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4916 (to save space, not time) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4917 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4918 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4919 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4920 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4921 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4922 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4923
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4924 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4925 int fixupper(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4926 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4927 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4928 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4929 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4930
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4931 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4932 register Py_UNICODE ch;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4933
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4934 ch = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4935 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4936 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4937 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4938 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4939 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4940 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4941
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4942 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4943 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4944
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4945 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4946 int fixlower(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4947 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4948 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4949 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4950 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4951
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4952 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4953 register Py_UNICODE ch;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4954
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4955 ch = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4956 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4957 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4958 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4959 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4960 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4961 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4962
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4963 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4964 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4965
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4966 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4967 int fixswapcase(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4968 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4969 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4970 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4971 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4972
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4973 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4974 if (Py_UNICODE_ISUPPER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4975 *s = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4976 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4977 } else if (Py_UNICODE_ISLOWER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4978 *s = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4979 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4980 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4981 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4982 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4983
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4984 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4985 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4986
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4987 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4988 int fixcapitalize(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4989 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4990 Py_ssize_t len = self->length;
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4991 Py_UNICODE *s = self->str;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4992 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4993
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4994 if (len == 0)
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4995 return 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4996 if (Py_UNICODE_ISLOWER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4997 *s = Py_UNICODE_TOUPPER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4998 status = 1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4999 }
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5000 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5001 while (--len > 0) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5002 if (Py_UNICODE_ISUPPER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5003 *s = Py_UNICODE_TOLOWER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5004 status = 1;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5005 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5006 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5007 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
5008 return status;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5009 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5010
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5011 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5012 int fixtitle(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5013 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5014 register Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5015 register Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5016 int previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5017
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5018 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5019 if (PyUnicode_GET_SIZE(self) == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5020 Py_UNICODE ch = Py_UNICODE_TOTITLE(*p);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5021 if (*p != ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5022 *p = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5023 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5024 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5025 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5026 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5027 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5028
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5029 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5030 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5031 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5032 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5033
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5034 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5035 *p = Py_UNICODE_TOLOWER(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5036 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5037 *p = Py_UNICODE_TOTITLE(ch);
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5038
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5039 if (Py_UNICODE_ISLOWER(ch) ||
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5040 Py_UNICODE_ISUPPER(ch) ||
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5041 Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5042 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5043 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5044 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5045 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5046 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5047 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5048
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5049 PyObject *
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5050 PyUnicode_Join(PyObject *separator, PyObject *seq)
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5051 {
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5052 PyObject *internal_separator = NULL;
33499
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc.
Skip Montanaro <skip@pobox.com>
parents: 33495
diff changeset
5053 const Py_UNICODE blank = ' ';
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc.
Skip Montanaro <skip@pobox.com>
parents: 33495
diff changeset
5054 const Py_UNICODE *sep = &blank;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5055 Py_ssize_t seplen = 1;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5056 PyUnicodeObject *res = NULL; /* the result */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5057 Py_ssize_t res_alloc = 100; /* # allocated bytes for string in res */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5058 Py_ssize_t res_used; /* # used bytes */
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5059 Py_UNICODE *res_p; /* pointer to free byte in res's string area */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5060 PyObject *fseq; /* PySequence_Fast(seq) */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5061 Py_ssize_t seqlen; /* len(fseq) -- number of items in sequence */
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5062 PyObject *item;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5063 Py_ssize_t i;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5064
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5065 fseq = PySequence_Fast(seq, "");
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5066 if (fseq == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5067 return NULL;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5068 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5069
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5070 /* Grrrr. A codec may be invoked to convert str objects to
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5071 * Unicode, and so it's possible to call back into Python code
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5072 * during PyUnicode_FromObject(), and so it's possible for a sick
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5073 * codec to change the size of fseq (if seq is a list). Therefore
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5074 * we have to keep refetching the size -- can't assume seqlen
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5075 * is invariant.
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5076 */
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5077 seqlen = PySequence_Fast_GET_SIZE(fseq);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5078 /* If empty sequence, return u"". */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5079 if (seqlen == 0) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5080 res = _PyUnicode_New(0); /* empty sequence; return u"" */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5081 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5082 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5083 /* If singleton sequence with an exact Unicode, return that. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5084 if (seqlen == 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5085 item = PySequence_Fast_GET_ITEM(fseq, 0);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5086 if (PyUnicode_CheckExact(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5087 Py_INCREF(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5088 res = (PyUnicodeObject *)item;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5089 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5090 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5091 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5092
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5093 /* At least two items to join, or one that isn't exact Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5094 if (seqlen > 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5095 /* Set up sep and seplen -- they're needed. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5096 if (separator == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5097 sep = &blank;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5098 seplen = 1;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5099 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5100 else {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5101 internal_separator = PyUnicode_FromObject(separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5102 if (internal_separator == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5103 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5104 sep = PyUnicode_AS_UNICODE(internal_separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5105 seplen = PyUnicode_GET_SIZE(internal_separator);
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5106 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5107 seqlen = PySequence_Fast_GET_SIZE(fseq);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5108 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5109 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5110
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5111 /* Get space. */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5112 res = _PyUnicode_New(res_alloc);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5113 if (res == NULL)
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5114 goto onError;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5115 res_p = PyUnicode_AS_UNICODE(res);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5116 res_used = 0;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5117
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5118 for (i = 0; i < seqlen; ++i) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5119 Py_ssize_t itemlen;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5120 Py_ssize_t new_res_used;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5121
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5122 item = PySequence_Fast_GET_ITEM(fseq, i);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5123 /* Convert item to Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5124 if (! PyUnicode_Check(item) && ! PyString_Check(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5125 PyErr_Format(PyExc_TypeError,
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5126 "sequence item %zd: expected string or Unicode,"
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5127 " %.80s found",
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
5128 i, Py_Type(item)->tp_name);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
5129 goto onError;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
5130 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5131 item = PyUnicode_FromObject(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5132 if (item == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5133 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5134 /* We own a reference to item from here on. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5135
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5136 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5137 seqlen = PySequence_Fast_GET_SIZE(fseq);
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5138
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5139 /* Make sure we have enough space for the separator and the item. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5140 itemlen = PyUnicode_GET_SIZE(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5141 new_res_used = res_used + itemlen;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
5142 if (new_res_used < 0)
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5143 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5144 if (i < seqlen - 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5145 new_res_used += seplen;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
5146 if (new_res_used < 0)
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5147 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5148 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5149 if (new_res_used > res_alloc) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5150 /* double allocated size until it's big enough */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5151 do {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5152 res_alloc += res_alloc;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5153 if (res_alloc <= 0)
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5154 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5155 } while (new_res_used > res_alloc);
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5156 if (_PyUnicode_Resize(&res, res_alloc) < 0) {
20007
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
5157 Py_DECREF(item);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
5158 goto onError;
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
5159 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5160 res_p = PyUnicode_AS_UNICODE(res) + res_used;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5161 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5162
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5163 /* Copy item, and maybe the separator. */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5164 Py_UNICODE_COPY(res_p, PyUnicode_AS_UNICODE(item), itemlen);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5165 res_p += itemlen;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5166 if (i < seqlen - 1) {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5167 Py_UNICODE_COPY(res_p, sep, seplen);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5168 res_p += seplen;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5169 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5170 Py_DECREF(item);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5171 res_used = new_res_used;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5172 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5173
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5174 /* Shrink res to match the used area; this probably can't fail,
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5175 * but it's cheap to check.
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5176 */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5177 if (_PyUnicode_Resize(&res, res_used) < 0)
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5178 goto onError;
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5179
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5180 Done:
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5181 Py_XDECREF(internal_separator);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5182 Py_DECREF(fseq);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5183 return (PyObject *)res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5184
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5185 Overflow:
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5186 PyErr_SetString(PyExc_OverflowError,
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
5187 "join() result is too long for a Python string");
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5188 Py_DECREF(item);
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5189 /* fall through */
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5190
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5191 onError:
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5192 Py_XDECREF(internal_separator);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5193 Py_DECREF(fseq);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
5194 Py_XDECREF(res);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5195 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5196 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5197
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5198 static
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5199 PyUnicodeObject *pad(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5200 Py_ssize_t left,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5201 Py_ssize_t right,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5202 Py_UNICODE fill)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5203 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5204 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5205
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5206 if (left < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5207 left = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5208 if (right < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5209 right = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5210
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5211 if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5212 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5213 return self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5214 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5215
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5216 u = _PyUnicode_New(left + self->length + right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5217 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5218 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5219 Py_UNICODE_FILL(u->str, fill, left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5220 Py_UNICODE_COPY(u->str + left, self->str, self->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5221 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5222 Py_UNICODE_FILL(u->str + left + self->length, fill, right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5223 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5224
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5225 return u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5226 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5227
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5228 #define SPLIT_APPEND(data, left, right) \
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5229 str = PyUnicode_FromUnicode((data) + (left), (right) - (left)); \
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5230 if (!str) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5231 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5232 if (PyList_Append(list, str)) { \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5233 Py_DECREF(str); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5234 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5235 } \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5236 else \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5237 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5239 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5240 PyObject *split_whitespace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5241 PyObject *list,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5242 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5243 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5244 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5245 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5246 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5247 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5248
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5249 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5250 /* find a token */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5251 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5252 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5253 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5254 while (i < len && !Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5255 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5256 if (j < i) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5257 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5258 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5259 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5260 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5261 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5262 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5264 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5265 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5266 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5267 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5268 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5269
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5270 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5271 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5272 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5273 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5274
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5275 PyObject *PyUnicode_Splitlines(PyObject *string,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5276 int keepends)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5277 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5278 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5279 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5280 Py_ssize_t len;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5281 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5282 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5283 Py_UNICODE *data;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5284
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5285 string = PyUnicode_FromObject(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5286 if (string == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5287 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5288 data = PyUnicode_AS_UNICODE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5289 len = PyUnicode_GET_SIZE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5290
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5291 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5292 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5293 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5294
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5295 for (i = j = 0; i < len; ) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5296 Py_ssize_t eol;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5297
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5298 /* Find a line and append it */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5299 while (i < len && !BLOOM_LINEBREAK(data[i]))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5300 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5301
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5302 /* Skip the line break reading CRLF as one line break */
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5303 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5304 if (i < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5305 if (data[i] == '\r' && i + 1 < len &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5306 data[i+1] == '\n')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5307 i += 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5308 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5309 i++;
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5310 if (keepends)
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5311 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5312 }
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5313 SPLIT_APPEND(data, j, eol);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5314 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5315 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5316 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5317 SPLIT_APPEND(data, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5318 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5319
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5320 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5321 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5322
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5323 onError:
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
5324 Py_XDECREF(list);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5325 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5326 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5327 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5328
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5329 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5330 PyObject *split_char(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5331 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5332 Py_UNICODE ch,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5333 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5334 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5335 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5336 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5337 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5338 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5340 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5341 if (self->str[i] == ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5342 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5343 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5344 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5345 i = j = i + 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5346 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5347 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5348 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5349 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5350 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5351 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5352 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5353
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5354 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5355 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5356 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5357 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5358
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5359 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5360 PyObject *split_substring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5361 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5362 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5363 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5364 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5365 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5366 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5367 Py_ssize_t len = self->length;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5368 Py_ssize_t sublen = substring->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5369 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5370
15609
0906c8e2c961 Fix off-by-one error in split_substring(). Fixes SF bug #122162.
Guido van Rossum <guido@python.org>
parents: 15579
diff changeset
5371 for (i = j = 0; i <= len - sublen; ) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5372 if (Py_UNICODE_MATCH(self, i, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5373 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5374 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5375 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5376 i = j = i + sublen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5377 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5378 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5379 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5380 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5381 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5382 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5383 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5384
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5385 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5386 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5387 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5388 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5389
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5390 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5391 PyObject *rsplit_whitespace(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5392 PyObject *list,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5393 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5394 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5395 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5396 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5397 Py_ssize_t len = self->length;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5398 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5399
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5400 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5401 /* find a token */
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5402 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5403 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5404 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5405 while (i >= 0 && !Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5406 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5407 if (j > i) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5408 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5409 break;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5410 SPLIT_APPEND(self->str, i + 1, j + 1);
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5411 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5412 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5413 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5414 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5415 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5416 if (j >= 0) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5417 SPLIT_APPEND(self->str, 0, j + 1);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5418 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5419 if (PyList_Reverse(list) < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5420 goto onError;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5421 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5422
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5423 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5424 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5425 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5426 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5427
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5428 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5429 PyObject *rsplit_char(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5430 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5431 Py_UNICODE ch,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5432 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5433 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5434 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5435 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5436 Py_ssize_t len = self->length;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5437 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5438
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5439 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5440 if (self->str[i] == ch) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5441 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5442 break;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5443 SPLIT_APPEND(self->str, i + 1, j + 1);
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5444 j = i = i - 1;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5445 } else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5446 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5447 }
31252
892116a886e0 Fix unicode.rsplit()'s bug that ignores separater on the end of string when
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31248
diff changeset
5448 if (j >= -1) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5449 SPLIT_APPEND(self->str, 0, j + 1);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5450 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5451 if (PyList_Reverse(list) < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5452 goto onError;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5453 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5454
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5455 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5456 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5457 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5458 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5459
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5460 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5461 PyObject *rsplit_substring(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5462 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5463 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5464 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5465 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5466 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5467 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5468 Py_ssize_t len = self->length;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5469 Py_ssize_t sublen = substring->length;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5470 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5471
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5472 for (i = len - sublen, j = len; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5473 if (Py_UNICODE_MATCH(self, i, substring)) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5474 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5475 break;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5476 SPLIT_APPEND(self->str, i + sublen, j);
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5477 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5478 i -= sublen;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5479 } else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5480 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5481 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5482 if (j >= 0) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5483 SPLIT_APPEND(self->str, 0, j);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5484 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5485 if (PyList_Reverse(list) < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5486 goto onError;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5487 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5488
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5489 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5490 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5491 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5492 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5493
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5494 #undef SPLIT_APPEND
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5495
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5496 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5497 PyObject *split(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5498 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5499 Py_ssize_t maxcount)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5500 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5501 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5502
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5503 if (maxcount < 0)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5504 maxcount = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5505
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5506 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5507 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5508 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5509
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5510 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5511 return split_whitespace(self,list,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5512
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5513 else if (substring->length == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5514 return split_char(self,list,substring->str[0],maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5515
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5516 else if (substring->length == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5517 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5518 PyErr_SetString(PyExc_ValueError, "empty separator");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5519 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5520 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5521 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5522 return split_substring(self,list,substring,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5523 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5524
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5525 static
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5526 PyObject *rsplit(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5527 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5528 Py_ssize_t maxcount)
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5529 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5530 PyObject *list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5531
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5532 if (maxcount < 0)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5533 maxcount = PY_SSIZE_T_MAX;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5534
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5535 list = PyList_New(0);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5536 if (!list)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5537 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5538
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5539 if (substring == NULL)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5540 return rsplit_whitespace(self,list,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5541
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5542 else if (substring->length == 1)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5543 return rsplit_char(self,list,substring->str[0],maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5544
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5545 else if (substring->length == 0) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5546 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5547 PyErr_SetString(PyExc_ValueError, "empty separator");
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5548 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5549 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5550 else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5551 return rsplit_substring(self,list,substring,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5552 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5553
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5554 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5555 PyObject *replace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5556 PyUnicodeObject *str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5557 PyUnicodeObject *str2,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5558 Py_ssize_t maxcount)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5559 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5560 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5561
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5562 if (maxcount < 0)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5563 maxcount = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5564
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5565 if (str1->length == str2->length) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5566 /* same length */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5567 Py_ssize_t i;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5568 if (str1->length == 1) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5569 /* replace characters */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5570 Py_UNICODE u1, u2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5571 if (!findchar(self->str, self->length, str1->str[0]))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5572 goto nothing;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5573 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5574 if (!u)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5575 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5576 Py_UNICODE_COPY(u->str, self->str, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5577 u1 = str1->str[0];
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5578 u2 = str2->str[0];
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5579 for (i = 0; i < u->length; i++)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5580 if (u->str[i] == u1) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5581 if (--maxcount < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5582 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5583 u->str[i] = u2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5584 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5585 } else {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5586 i = fastsearch(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5587 self->str, self->length, str1->str, str1->length, FAST_SEARCH
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5588 );
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5589 if (i < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5590 goto nothing;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5591 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5592 if (!u)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5593 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5594 Py_UNICODE_COPY(u->str, self->str, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5595 while (i <= self->length - str1->length)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5596 if (Py_UNICODE_MATCH(self, i, str1)) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5597 if (--maxcount < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5598 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5599 Py_UNICODE_COPY(u->str+i, str2->str, str2->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5600 i += str1->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5601 } else
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5602 i++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5603 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5604 } else {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5605
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5606 Py_ssize_t n, i, j, e;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5607 Py_ssize_t product, new_size, delta;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5608 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5609
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5610 /* replace strings */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5611 n = stringlib_count(self->str, self->length, str1->str, str1->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5612 if (n > maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5613 n = maxcount;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5614 if (n == 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5615 goto nothing;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5616 /* new_size = self->length + n * (str2->length - str1->length)); */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5617 delta = (str2->length - str1->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5618 if (delta == 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5619 new_size = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5620 } else {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5621 product = n * (str2->length - str1->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5622 if ((product / (str2->length - str1->length)) != n) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5623 PyErr_SetString(PyExc_OverflowError,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5624 "replace string is too long");
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5625 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5626 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5627 new_size = self->length + product;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5628 if (new_size < 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5629 PyErr_SetString(PyExc_OverflowError,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5630 "replace string is too long");
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5631 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5632 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5633 }
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5634 u = _PyUnicode_New(new_size);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5635 if (!u)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5636 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5637 i = 0;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5638 p = u->str;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5639 e = self->length - str1->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5640 if (str1->length > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5641 while (n-- > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5642 /* look for next match */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5643 j = i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5644 while (j <= e) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5645 if (Py_UNICODE_MATCH(self, j, str1))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5646 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5647 j++;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5648 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5649 if (j > i) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5650 if (j > e)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5651 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5652 /* copy unchanged part [i:j] */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5653 Py_UNICODE_COPY(p, self->str+i, j-i);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5654 p += j - i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5655 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5656 /* copy substitution string */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5657 if (str2->length > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5658 Py_UNICODE_COPY(p, str2->str, str2->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5659 p += str2->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5660 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5661 i = j + str1->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5662 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5663 if (i < self->length)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5664 /* copy tail [i:] */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5665 Py_UNICODE_COPY(p, self->str+i, self->length-i);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5666 } else {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5667 /* interleave */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5668 while (n > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5669 Py_UNICODE_COPY(p, str2->str, str2->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5670 p += str2->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5671 if (--n <= 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5672 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5673 *p++ = self->str[i++];
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5674 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5675 Py_UNICODE_COPY(p, self->str+i, self->length-i);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5676 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5677 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5678 return (PyObject *) u;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5679
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5680 nothing:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5681 /* nothing to replace; return original string (when possible) */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5682 if (PyUnicode_CheckExact(self)) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5683 Py_INCREF(self);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5684 return (PyObject *) self;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5685 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5686 return PyUnicode_FromUnicode(self->str, self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5687 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5688
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5689 /* --- Unicode Object Methods --------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5690
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5691 PyDoc_STRVAR(title__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5692 "S.title() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5693 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5694 Return a titlecased version of S, i.e. words start with title case\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5695 characters, all remaining cased characters have lower case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5696
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5697 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5698 unicode_title(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5699 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5700 return fixup(self, fixtitle);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5701 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5702
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5703 PyDoc_STRVAR(capitalize__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5704 "S.capitalize() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5705 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5706 Return a capitalized version of S, i.e. make the first character\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5707 have upper case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5708
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5709 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5710 unicode_capitalize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5711 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5712 return fixup(self, fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5713 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5714
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5715 #if 0
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5716 PyDoc_STRVAR(capwords__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5717 "S.capwords() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5718 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5719 Apply .capitalize() to all words in S and return the result with\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5720 normalized whitespace (all whitespace strings are replaced by ' ').");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5721
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5722 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5723 unicode_capwords(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5724 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5725 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5726 PyObject *item;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5727 Py_ssize_t i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5728
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5729 /* Split into words */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5730 list = split(self, NULL, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5731 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5732 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5733
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5734 /* Capitalize each word */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5735 for (i = 0; i < PyList_GET_SIZE(list); i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5736 item = fixup((PyUnicodeObject *)PyList_GET_ITEM(list, i),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5737 fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5738 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5739 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5740 Py_DECREF(PyList_GET_ITEM(list, i));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5741 PyList_SET_ITEM(list, i, item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5742 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5743
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5744 /* Join the words to form a new string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5745 item = PyUnicode_Join(NULL, list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5746
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5747 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5748 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5749 return (PyObject *)item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5750 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5751 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5752
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5753 /* Argument converter. Coerces to a single unicode character */
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5754
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5755 static int
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5756 convert_uc(PyObject *obj, void *addr)
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5757 {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5758 Py_UNICODE *fillcharloc = (Py_UNICODE *)addr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5759 PyObject *uniobj;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5760 Py_UNICODE *unistr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5761
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5762 uniobj = PyUnicode_FromObject(obj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5763 if (uniobj == NULL) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5764 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5765 "The fill character cannot be converted to Unicode");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5766 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5767 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5768 if (PyUnicode_GET_SIZE(uniobj) != 1) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5769 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5770 "The fill character must be exactly one character long");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5771 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5772 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5773 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5774 unistr = PyUnicode_AS_UNICODE(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5775 *fillcharloc = unistr[0];
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5776 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5777 return 1;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5778 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5779
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5780 PyDoc_STRVAR(center__doc__,
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5781 "S.center(width[, fillchar]) -> unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5782 \n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5783 Return S centered in a Unicode string of length width. Padding is\n\
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5784 done using the specified fill character (default is a space)");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5785
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5786 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5787 unicode_center(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5788 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5789 Py_ssize_t marg, left;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5790 Py_ssize_t width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5791 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5792
36532
0c096be27000 Use correct PyArg_Parse format char for Py_ssize_t in unicode.center().
Thomas Wouters <thomas@python.org>
parents: 36519
diff changeset
5793 if (!PyArg_ParseTuple(args, "n|O&:center", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5794 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5795
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5796 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5797 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5798 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5799 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5800
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5801 marg = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5802 left = marg / 2 + (marg & width & 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5803
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5804 return (PyObject*) pad(self, left, marg - left, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5805 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5806
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5807 #if 0
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5808
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5809 /* This code should go into some future Unicode collation support
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5810 module. The basic comparison should compare ordinals on a naive
13946
7a99450da2d6 Clean up warning from Monterey compiler.
Trent Mick <trentm@activestate.com>
parents: 13890
diff changeset
5811 basis (this is what Java does and thus JPython too). */
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5812
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5813 /* speedy UTF-16 code point order comparison */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5814 /* gleaned from: */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5815 /* http://www-4.ibm.com/software/developer/library/utf16.html?dwzone=unicode */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5816
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
5817 static short utf16Fixup[32] =
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5818 {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5819 0, 0, 0, 0, 0, 0, 0, 0,
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5820 0, 0, 0, 0, 0, 0, 0, 0,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5821 0, 0, 0, 0, 0, 0, 0, 0,
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
5822 0, 0, 0, 0x2000, -0x800, -0x800, -0x800, -0x800
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5823 };
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5824
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5825 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5826 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5827 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5828 Py_ssize_t len1, len2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5829
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5830 Py_UNICODE *s1 = str1->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5831 Py_UNICODE *s2 = str2->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5832
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5833 len1 = str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5834 len2 = str2->length;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5835
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5836 while (len1 > 0 && len2 > 0) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5837 Py_UNICODE c1, c2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5838
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5839 c1 = *s1++;
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5840 c2 = *s2++;
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5841
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5842 if (c1 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5843 c1 += utf16Fixup[c1>>11];
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5844 if (c2 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5845 c2 += utf16Fixup[c2>>11];
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5846 /* now c1 and c2 are in UTF-32-compatible order */
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5847
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5848 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5849 return (c1 < c2) ? -1 : 1;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5850
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5851 len1--; len2--;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5852 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5853
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5854 return (len1 < len2) ? -1 : (len1 != len2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5855 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5856
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5857 #else
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5858
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5859 static int
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5860 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5861 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5862 register Py_ssize_t len1, len2;
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5863
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5864 Py_UNICODE *s1 = str1->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5865 Py_UNICODE *s2 = str2->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5866
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5867 len1 = str1->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5868 len2 = str2->length;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5869
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5870 while (len1 > 0 && len2 > 0) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5871 Py_UNICODE c1, c2;
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5872
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5873 c1 = *s1++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5874 c2 = *s2++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5875
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5876 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5877 return (c1 < c2) ? -1 : 1;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5878
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5879 len1--; len2--;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5880 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5881
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5882 return (len1 < len2) ? -1 : (len1 != len2);
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5883 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5884
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5885 #endif
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5886
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5887 int PyUnicode_Compare(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5888 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5889 {
41596
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5890 if (PyUnicode_Check(left) && PyUnicode_Check(right))
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5891 return unicode_compare((PyUnicodeObject *)left,
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5892 (PyUnicodeObject *)right);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5893 if ((PyString_Check(left) && PyUnicode_Check(right)) ||
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5894 (PyUnicode_Check(left) && PyString_Check(right))) {
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5895 if (PyUnicode_Check(left))
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5896 left = _PyUnicode_AsDefaultEncodedString(left, NULL);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5897 if (PyUnicode_Check(right))
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5898 right = _PyUnicode_AsDefaultEncodedString(right, NULL);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5899 assert(PyString_Check(left));
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5900 assert(PyString_Check(right));
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5901 return PyObject_Compare(left, right);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5902 }
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5903 PyErr_Format(PyExc_TypeError,
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5904 "Can't compare %.100s and %.100s",
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5905 left->ob_type->tp_name,
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5906 right->ob_type->tp_name);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5907 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5908 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5909
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5910 int
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5911 PyUnicode_CompareWithASCIIString(PyObject* uni, const char* str)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5912 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5913 int i;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5914 Py_UNICODE *id;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5915 assert(PyUnicode_Check(uni));
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5916 id = PyUnicode_AS_UNICODE(uni);
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5917 /* Compare Unicode string and source character set string */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5918 for (i = 0; id[i] && str[i]; i++)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5919 if (id[i] != str[i])
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5920 return ((int)id[i] < (int)str[i]) ? -1 : 1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5921 if (id[i])
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5922 return 1; /* uni is longer */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5923 if (str[i])
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5924 return -1; /* str is longer */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5925 return 0;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5926 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5927
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5928 PyObject *PyUnicode_RichCompare(PyObject *left,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5929 PyObject *right,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5930 int op)
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5931 {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5932 int result;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5933
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5934 result = PyUnicode_Compare(left, right);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5935 if (result == -1 && PyErr_Occurred())
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5936 goto onError;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5937
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5938 /* Convert the return value to a Boolean */
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5939 switch (op) {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5940 case Py_EQ:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5941 result = (result == 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5942 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5943 case Py_NE:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5944 result = (result != 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5945 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5946 case Py_LE:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5947 result = (result <= 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5948 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5949 case Py_GE:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5950 result = (result >= 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5951 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5952 case Py_LT:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5953 result = (result == -1);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5954 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5955 case Py_GT:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5956 result = (result == 1);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5957 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5958 }
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5959 return PyBool_FromLong(result);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5960
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5961 onError:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5962
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5963 /* Standard case
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5964
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5965 Type errors mean that PyUnicode_FromObject() could not convert
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5966 one of the arguments (usually the right hand side) to Unicode,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5967 ie. we can't handle the comparison request. However, it is
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5968 possible that the other object knows a comparison method, which
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5969 is why we return Py_NotImplemented to give the other object a
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5970 chance.
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5971
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5972 */
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5973 if (PyErr_ExceptionMatches(PyExc_TypeError)) {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5974 PyErr_Clear();
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5975 Py_INCREF(Py_NotImplemented);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5976 return Py_NotImplemented;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5977 }
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5978 if (op != Py_EQ && op != Py_NE)
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5979 return NULL;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5980
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5981 /* Equality comparison.
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5982
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5983 This is a special case: we silence any PyExc_UnicodeDecodeError
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5984 and instead turn it into a PyErr_UnicodeWarning.
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5985
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5986 */
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5987 if (!PyErr_ExceptionMatches(PyExc_UnicodeDecodeError))
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5988 return NULL;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5989 PyErr_Clear();
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5990 if (PyErr_Warn(PyExc_UnicodeWarning,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5991 (op == Py_EQ) ?
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5992 "Unicode equal comparison "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5993 "failed to convert both arguments to Unicode - "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5994 "interpreting them as being unequal" :
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5995 "Unicode unequal comparison "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5996 "failed to convert both arguments to Unicode - "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5997 "interpreting them as being unequal"
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5998 ) < 0)
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5999 return NULL;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
6000 result = (op == Py_NE);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
6001 return PyBool_FromLong(result);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
6002 }
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
6003
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6004 int PyUnicode_Contains(PyObject *container,
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6005 PyObject *element)
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6006 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6007 PyObject *str, *sub;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6008 int result;
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6009
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6010 /* Coerce the two arguments */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6011 sub = PyUnicode_FromObject(element);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6012 if (!sub) {
42065
4877008074a9 Mention name of left operand, if "foo in unicode_string"
Walter Dörwald <walter@livinglogic.de>
parents: 42047
diff changeset
6013 PyErr_Format(PyExc_TypeError,
4877008074a9 Mention name of left operand, if "foo in unicode_string"
Walter Dörwald <walter@livinglogic.de>
parents: 42047
diff changeset
6014 "'in <string>' requires string as left operand, not %s",
4877008074a9 Mention name of left operand, if "foo in unicode_string"
Walter Dörwald <walter@livinglogic.de>
parents: 42047
diff changeset
6015 element->ob_type->tp_name);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6016 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6017 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6018
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6019 str = PyUnicode_FromObject(container);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6020 if (!str) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6021 Py_DECREF(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6022 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6023 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6024
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6025 result = stringlib_contains_obj(str, sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6026
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6027 Py_DECREF(str);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6028 Py_DECREF(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6029
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6030 return result;
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6031 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6032
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6033 /* Concat to string or Unicode object giving a new Unicode object. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6034
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6035 PyObject *PyUnicode_Concat(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6036 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6037 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6038 PyUnicodeObject *u = NULL, *v = NULL, *w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6039
41451
e2d7a4c39360 Disallow u"..." + b"..." and b"..." + u"...".
Guido van Rossum <guido@python.org>
parents: 40909
diff changeset
6040 if (PyBytes_Check(left) || PyBytes_Check(right))
e2d7a4c39360 Disallow u"..." + b"..." and b"..." + u"...".
Guido van Rossum <guido@python.org>
parents: 40909
diff changeset
6041 return PyBytes_Concat(left, right);
e2d7a4c39360 Disallow u"..." + b"..." and b"..." + u"...".
Guido van Rossum <guido@python.org>
parents: 40909
diff changeset
6042
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6043 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6044 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6045 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6046 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6047 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6048 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6049 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6050
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6051 /* Shortcuts */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6052 if (v == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6053 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6054 return (PyObject *)u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6055 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6056 if (u == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6057 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6058 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6059 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6060
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6061 /* Concat the two Unicode strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6062 w = _PyUnicode_New(u->length + v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6063 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6064 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6065 Py_UNICODE_COPY(w->str, u->str, u->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6066 Py_UNICODE_COPY(w->str + u->length, v->str, v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6067
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6068 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6069 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6070 return (PyObject *)w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6071
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6072 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6073 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6074 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6075 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6076 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6077
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6078 void
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6079 PyUnicode_Append(PyObject **pleft, PyObject *right)
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6080 {
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6081 PyObject *new;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6082 if (*pleft == NULL)
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6083 return;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6084 if (right == NULL || !PyUnicode_Check(*pleft)) {
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6085 Py_DECREF(*pleft);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6086 *pleft = NULL;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6087 return;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6088 }
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6089 new = PyUnicode_Concat(*pleft, right);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6090 Py_DECREF(*pleft);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6091 *pleft = new;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6092 }
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6093
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6094 void
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6095 PyUnicode_AppendAndDel(PyObject **pleft, PyObject *right)
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6096 {
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6097 PyUnicode_Append(pleft, right);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6098 Py_XDECREF(right);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6099 }
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6100
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6101 PyDoc_STRVAR(count__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6102 "S.count(sub[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6103 \n\
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6104 Return the number of non-overlapping occurrences of substring sub in\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6105 Unicode string S[start:end]. Optional arguments start and end are\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6106 interpreted as in slice notation.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6107
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6108 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6109 unicode_count(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6110 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6111 PyUnicodeObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6112 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6113 Py_ssize_t end = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6114 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6115
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6116 if (!PyArg_ParseTuple(args, "O|O&O&:count", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6117 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6118 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6119
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6120 substring = (PyUnicodeObject *)PyUnicode_FromObject(
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6121 (PyObject *)substring);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6122 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6123 return NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6124
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6125 FIX_START_END(self);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6126
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6127 result = PyInt_FromSsize_t(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6128 stringlib_count(self->str + start, end - start,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6129 substring->str, substring->length)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6130 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6131
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6132 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6133
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6134 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6135 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6136
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6137 PyDoc_STRVAR(encode__doc__,
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6138 "S.encode([encoding[,errors]]) -> string or unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6139 \n\
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6140 Encodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6141 to the default encoding. errors may be given to set a different error\n\
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6142 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
6143 a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
6144 'xmlcharrefreplace' as well as any other name registered with\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
6145 codecs.register_error that can handle UnicodeEncodeErrors.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6146
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6147 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6148 unicode_encode(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6149 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6150 char *encoding = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6151 char *errors = NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6152 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6153
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6154 if (!PyArg_ParseTuple(args, "|ss:encode", &encoding, &errors))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6155 return NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6156 v = PyUnicode_AsEncodedObject((PyObject *)self, encoding, errors);
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6157 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6158 goto onError;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
6159 if (!PyBytes_Check(v)) {
41598
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6160 if (PyString_Check(v)) {
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6161 /* Old codec, turn it into bytes */
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6162 PyObject *b = PyBytes_FromObject(v);
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6163 Py_DECREF(v);
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6164 return b;
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6165 }
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6166 PyErr_Format(PyExc_TypeError,
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
6167 "encoder did not return a bytes object "
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6168 "(type=%.400s)",
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
6169 Py_Type(v)->tp_name);
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6170 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6171 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6172 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6173 return v;
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6174
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6175 onError:
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6176 return NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6177 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6178
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6179 PyDoc_STRVAR(decode__doc__,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6180 "S.decode([encoding[,errors]]) -> string or unicode\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6181 \n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6182 Decodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6183 to the default encoding. errors may be given to set a different error\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6184 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6185 a UnicodeDecodeError. Other possible values are 'ignore' and 'replace'\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6186 as well as any other name registerd with codecs.register_error that is\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6187 able to handle UnicodeDecodeErrors.");
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6188
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6189 static PyObject *
32571
bec0c7618625 Fix a copy&paste typo.
Marc-André Lemburg <mal@egenix.com>
parents: 32552
diff changeset
6190 unicode_decode(PyUnicodeObject *self, PyObject *args)
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6191 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6192 char *encoding = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6193 char *errors = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6194 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6195
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6196 if (!PyArg_ParseTuple(args, "|ss:decode", &encoding, &errors))
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6197 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6198 v = PyUnicode_AsDecodedObject((PyObject *)self, encoding, errors);
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6199 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6200 goto onError;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6201 if (!PyString_Check(v) && !PyUnicode_Check(v)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6202 PyErr_Format(PyExc_TypeError,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6203 "decoder did not return a string/unicode object "
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6204 "(type=%.400s)",
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
6205 Py_Type(v)->tp_name);
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6206 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6207 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6208 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6209 return v;
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6210
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6211 onError:
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6212 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6213 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6214
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6215 PyDoc_STRVAR(expandtabs__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6216 "S.expandtabs([tabsize]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6217 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6218 Return a copy of S where all tab characters are expanded using spaces.\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6219 If tabsize is not given, a tab size of 8 characters is assumed.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6220
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6221 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6222 unicode_expandtabs(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6223 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6224 Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6225 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6226 Py_UNICODE *q;
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6227 Py_ssize_t i, j, old_j;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6228 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6229 int tabsize = 8;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6230
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6231 if (!PyArg_ParseTuple(args, "|i:expandtabs", &tabsize))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6232 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6233
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13576
diff changeset
6234 /* First pass: determine size of output string */
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6235 i = j = old_j = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6236 e = self->str + self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6237 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6238 if (*p == '\t') {
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6239 if (tabsize > 0) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6240 j += tabsize - (j % tabsize);
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6241 if (old_j > j) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6242 PyErr_SetString(PyExc_OverflowError,
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6243 "new string is too long");
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6244 return NULL;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6245 }
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6246 old_j = j;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6247 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6248 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6249 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6250 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6251 if (*p == '\n' || *p == '\r') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6252 i += j;
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6253 old_j = j = 0;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6254 if (i < 0) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6255 PyErr_SetString(PyExc_OverflowError,
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6256 "new string is too long");
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6257 return NULL;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6258 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6259 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6260 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6261
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6262 if ((i + j) < 0) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6263 PyErr_SetString(PyExc_OverflowError, "new string is too long");
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6264 return NULL;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6265 }
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6266
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6267 /* Second pass: create output string and fill it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6268 u = _PyUnicode_New(i + j);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6269 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6270 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6271
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6272 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6273 q = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6274
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6275 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6276 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6277 if (tabsize > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6278 i = tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6279 j += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6280 while (i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6281 *q++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6282 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6283 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6284 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6285 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6286 *q++ = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6287 if (*p == '\n' || *p == '\r')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6288 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6289 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6290
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6291 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6292 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6293
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6294 PyDoc_STRVAR(find__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6295 "S.find(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6296 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6297 Return the lowest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6298 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6299 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6300 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6301 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6302
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6303 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6304 unicode_find(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6305 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6306 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6307 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6308 Py_ssize_t end = PY_SSIZE_T_MAX;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6309 Py_ssize_t result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6310
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6311 if (!PyArg_ParseTuple(args, "O|O&O&:find", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6312 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6313 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6314 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6315 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6316 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6317
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6318 result = stringlib_find_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6319 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6320 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6321 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6322 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6323
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6324 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6325
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6326 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6327 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6328
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6329 static PyObject *
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6330 unicode_getitem(PyUnicodeObject *self, Py_ssize_t index)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6331 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6332 if (index < 0 || index >= self->length) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6333 PyErr_SetString(PyExc_IndexError, "string index out of range");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6334 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6335 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6336
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6337 return (PyObject*) PyUnicode_FromUnicode(&self->str[index], 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6338 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6340 static long
41596
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6341 unicode_hash(PyObject *self)
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6342 {
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6343 /* Since Unicode objects compare equal to their UTF-8 string
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6344 counterparts, we hash the UTF-8 string. */
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6345 PyObject *v = _PyUnicode_AsDefaultEncodedString(self, NULL);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6346 return PyObject_Hash(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6347 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6348
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6349 PyDoc_STRVAR(index__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6350 "S.index(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6351 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6352 Like S.find() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6353
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6354 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6355 unicode_index(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6356 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6357 Py_ssize_t result;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6358 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6359 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6360 Py_ssize_t end = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6361
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6362 if (!PyArg_ParseTuple(args, "O|O&O&:index", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6363 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6364 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6365 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6366 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6367 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6368
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6369 result = stringlib_find_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6370 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6371 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6372 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6373 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6374
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6375 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6376
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6377 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6378 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6379 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6380 }
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6381
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6382 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6383 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6384
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6385 PyDoc_STRVAR(islower__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6386 "S.islower() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6387 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6388 Return True if all cased characters in S are lowercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6389 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6390
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6391 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6392 unicode_islower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6393 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6394 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6395 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6396 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6397
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6398 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6399 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6400 return PyBool_FromLong(Py_UNICODE_ISLOWER(*p));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6401
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6402 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6403 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6404 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6405
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6406 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6407 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6408 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6409 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6410
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6411 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6412 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6413 else if (!cased && Py_UNICODE_ISLOWER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6414 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6415 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6416 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6417 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6418
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6419 PyDoc_STRVAR(isupper__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6420 "S.isupper() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6421 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6422 Return True if all cased characters in S are uppercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6423 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6424
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6425 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6426 unicode_isupper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6427 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6428 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6429 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6430 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6431
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6432 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6433 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6434 return PyBool_FromLong(Py_UNICODE_ISUPPER(*p) != 0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6435
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6436 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6437 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6438 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6439
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6440 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6441 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6442 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6443 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6444
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6445 if (Py_UNICODE_ISLOWER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6446 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6447 else if (!cased && Py_UNICODE_ISUPPER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6448 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6449 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6450 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6451 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6452
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6453 PyDoc_STRVAR(istitle__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6454 "S.istitle() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6455 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6456 Return True if S is a titlecased string and there is at least one\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6457 character in S, i.e. upper- and titlecase characters may only\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6458 follow uncased characters and lowercase characters only cased ones.\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6459 Return False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6460
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6461 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6462 unicode_istitle(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6463 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6464 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6465 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6466 int cased, previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6467
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6468 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6469 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6470 return PyBool_FromLong((Py_UNICODE_ISTITLE(*p) != 0) ||
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6471 (Py_UNICODE_ISUPPER(*p) != 0));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6472
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6473 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6474 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6475 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6476
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6477 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6478 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6479 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6480 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6481 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6482
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6483 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6484 if (previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6485 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6486 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6487 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6488 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6489 else if (Py_UNICODE_ISLOWER(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6490 if (!previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6491 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6492 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6493 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6494 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6495 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6496 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6497 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6498 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6499 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6500
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6501 PyDoc_STRVAR(isspace__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6502 "S.isspace() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6503 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6504 Return True if all characters in S are whitespace\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6505 and there is at least one character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6506
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6507 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6508 unicode_isspace(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6509 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6510 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6511 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6512
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6513 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6514 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6515 Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6516 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6517
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6518 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6519 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6520 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6521
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6522 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6523 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6524 if (!Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6525 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6526 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6527 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6528 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6529
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6530 PyDoc_STRVAR(isalpha__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6531 "S.isalpha() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6532 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6533 Return True if all characters in S are alphabetic\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6534 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6535
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6536 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6537 unicode_isalpha(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6538 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6539 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6540 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6541
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6542 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6543 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6544 Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6545 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6546
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6547 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6548 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6549 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6550
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6551 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6552 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6553 if (!Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6554 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6555 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6556 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6557 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6558
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6559 PyDoc_STRVAR(isalnum__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6560 "S.isalnum() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6561 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6562 Return True if all characters in S are alphanumeric\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6563 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6564
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6565 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6566 unicode_isalnum(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6567 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6568 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6569 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6570
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6571 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6572 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6573 Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6574 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6575
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6576 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6577 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6578 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6579
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6580 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6581 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6582 if (!Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6583 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6584 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6585 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6586 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6587
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6588 PyDoc_STRVAR(isdecimal__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6589 "S.isdecimal() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6590 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6591 Return True if there are only decimal characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6592 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6593
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6594 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6595 unicode_isdecimal(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6596 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6597 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6598 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6599
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6600 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6601 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6602 Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6603 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6604
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6605 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6606 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6607 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6608
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6609 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6610 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6611 if (!Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6612 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6613 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6614 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6615 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6616
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6617 PyDoc_STRVAR(isdigit__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6618 "S.isdigit() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6619 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6620 Return True if all characters in S are digits\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6621 and there is at least one character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6622
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6623 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6624 unicode_isdigit(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6625 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6626 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6627 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6628
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6629 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6630 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6631 Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6632 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6633
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6634 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6635 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6636 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6637
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6638 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6639 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6640 if (!Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6641 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6642 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6643 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6644 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6645
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6646 PyDoc_STRVAR(isnumeric__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6647 "S.isnumeric() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6648 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6649 Return True if there are only numeric characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6650 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6651
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6652 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6653 unicode_isnumeric(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6654 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6655 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6656 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6657
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6658 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6659 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6660 Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6661 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6662
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6663 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6664 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6665 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6666
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6667 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6668 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6669 if (!Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6670 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6671 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6672 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6673 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6674
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6675 PyDoc_STRVAR(join__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6676 "S.join(sequence) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6677 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6678 Return a string which is the concatenation of the strings in the\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6679 sequence. The separator between elements is S.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6680
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6681 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6682 unicode_join(PyObject *self, PyObject *data)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6683 {
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6684 return PyUnicode_Join(self, data);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6685 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6686
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6687 static Py_ssize_t
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6688 unicode_length(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6689 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6690 return self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6691 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6692
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6693 PyDoc_STRVAR(ljust__doc__,
32197
3a84e23b042f - SF #962502: Add two more methods for unicode type; width() and
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31873
diff changeset
6694 "S.ljust(width[, fillchar]) -> int\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6695 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6696 Return S left justified in a Unicode string of length width. Padding is\n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6697 done using the specified fill character (default is a space).");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6698
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6699 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6700 unicode_ljust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6701 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6702 Py_ssize_t width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6703 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6704
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6705 if (!PyArg_ParseTuple(args, "n|O&:ljust", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6706 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6707
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
6708 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6709 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6710 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6711 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6712
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6713 return (PyObject*) pad(self, 0, width - self->length, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6714 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6715
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6716 PyDoc_STRVAR(lower__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6717 "S.lower() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6718 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6719 Return a copy of the string S converted to lowercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6720
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6721 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6722 unicode_lower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6723 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6724 return fixup(self, fixlower);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6725 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6726
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6727 #define LEFTSTRIP 0
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6728 #define RIGHTSTRIP 1
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6729 #define BOTHSTRIP 2
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6730
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6731 /* Arrays indexed by above */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6732 static const char *stripformat[] = {"|O:lstrip", "|O:rstrip", "|O:strip"};
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6733
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6734 #define STRIPNAME(i) (stripformat[i]+3)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6735
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6736 /* externally visible for str.strip(unicode) */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6737 PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6738 _PyUnicode_XStrip(PyUnicodeObject *self, int striptype, PyObject *sepobj)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6739 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6740 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6741 Py_ssize_t len = PyUnicode_GET_SIZE(self);
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6742 Py_UNICODE *sep = PyUnicode_AS_UNICODE(sepobj);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6743 Py_ssize_t seplen = PyUnicode_GET_SIZE(sepobj);
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6744 Py_ssize_t i, j;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6745
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6746 BLOOM_MASK sepmask = make_bloom_mask(sep, seplen);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6747
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6748 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6749 if (striptype != RIGHTSTRIP) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6750 while (i < len && BLOOM_MEMBER(sepmask, s[i], sep, seplen)) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6751 i++;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6752 }
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6753 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6754
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6755 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6756 if (striptype != LEFTSTRIP) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6757 do {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6758 j--;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6759 } while (j >= i && BLOOM_MEMBER(sepmask, s[j], sep, seplen));
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6760 j++;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6761 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6762
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6763 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6764 Py_INCREF(self);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6765 return (PyObject*)self;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6766 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6767 else
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6768 return PyUnicode_FromUnicode(s+i, j-i);
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6769 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6770
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6771
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6772 static PyObject *
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6773 do_strip(PyUnicodeObject *self, int striptype)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6774 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6775 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6776 Py_ssize_t len = PyUnicode_GET_SIZE(self), i, j;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6777
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6778 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6779 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6780 while (i < len && Py_UNICODE_ISSPACE(s[i])) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6781 i++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6782 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6783 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6784
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6785 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6786 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6787 do {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6788 j--;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6789 } while (j >= i && Py_UNICODE_ISSPACE(s[j]));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6790 j++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6791 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6792
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6793 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6794 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6795 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6796 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6797 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6798 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6799 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6800
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6801
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6802 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6803 do_argstrip(PyUnicodeObject *self, int striptype, PyObject *args)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6804 {
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6805 PyObject *sep = NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6806
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6807 if (!PyArg_ParseTuple(args, (char *)stripformat[striptype], &sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6808 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6809
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6810 if (sep != NULL && sep != Py_None) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6811 if (PyUnicode_Check(sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6812 return _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6813 else if (PyString_Check(sep)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6814 PyObject *res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6815 sep = PyUnicode_FromObject(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6816 if (sep==NULL)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6817 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6818 res = _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6819 Py_DECREF(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6820 return res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6821 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6822 else {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6823 PyErr_Format(PyExc_TypeError,
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6824 "%s arg must be None, unicode or str",
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6825 STRIPNAME(striptype));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6826 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6827 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6828 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6829
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6830 return do_strip(self, striptype);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6831 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6832
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6833
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6834 PyDoc_STRVAR(strip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6835 "S.strip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6836 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6837 Return a copy of the string S with leading and trailing\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6838 whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6839 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6840 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6841
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6842 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6843 unicode_strip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6844 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6845 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6846 return do_strip(self, BOTHSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6847 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6848 return do_argstrip(self, BOTHSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6849 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6850
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6851
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6852 PyDoc_STRVAR(lstrip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6853 "S.lstrip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6854 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6855 Return a copy of the string S with leading whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6856 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6857 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6858
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6859 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6860 unicode_lstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6861 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6862 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6863 return do_strip(self, LEFTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6864 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6865 return do_argstrip(self, LEFTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6866 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6867
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6868
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6869 PyDoc_STRVAR(rstrip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6870 "S.rstrip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6871 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6872 Return a copy of the string S with trailing whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6873 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6874 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6875
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6876 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6877 unicode_rstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6878 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6879 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6880 return do_strip(self, RIGHTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6881 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6882 return do_argstrip(self, RIGHTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6883 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6884
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6885
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6886 static PyObject*
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6887 unicode_repeat(PyUnicodeObject *str, Py_ssize_t len)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6888 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6889 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6890 Py_UNICODE *p;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6891 Py_ssize_t nchars;
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6892 size_t nbytes;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6893
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6894 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6895 len = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6896
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
6897 if (len == 1 && PyUnicode_CheckExact(str)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6898 /* no repeat, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6899 Py_INCREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6900 return (PyObject*) str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6901 }
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6902
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6903 /* ensure # of chars needed doesn't overflow int and # of bytes
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6904 * needed doesn't overflow size_t
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6905 */
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6906 nchars = len * str->length;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6907 if (len && nchars / len != str->length) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6908 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6909 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6910 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6911 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6912 nbytes = (nchars + 1) * sizeof(Py_UNICODE);
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6913 if (nbytes / sizeof(Py_UNICODE) != (size_t)(nchars + 1)) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6914 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6915 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6916 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6917 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6918 u = _PyUnicode_New(nchars);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6919 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6920 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6921
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6922 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6923
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6924 if (str->length == 1 && len > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6925 Py_UNICODE_FILL(p, str->str[0], len);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6926 } else {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6927 Py_ssize_t done = 0; /* number of characters copied this far */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6928 if (done < nchars) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6929 Py_UNICODE_COPY(p, str->str, str->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6930 done = str->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6931 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6932 while (done < nchars) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6933 int n = (done <= nchars-done) ? done : nchars-done;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6934 Py_UNICODE_COPY(p+done, p, n);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6935 done += n;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6936 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6937 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6938
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6939 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6940 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6941
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6942 PyObject *PyUnicode_Replace(PyObject *obj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6943 PyObject *subobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6944 PyObject *replobj,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6945 Py_ssize_t maxcount)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6946 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6947 PyObject *self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6948 PyObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6949 PyObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6950 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6951
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6952 self = PyUnicode_FromObject(obj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6953 if (self == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6954 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6955 str1 = PyUnicode_FromObject(subobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6956 if (str1 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6957 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6958 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6959 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6960 str2 = PyUnicode_FromObject(replobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6961 if (str2 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6962 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6963 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6964 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6965 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6966 result = replace((PyUnicodeObject *)self,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6967 (PyUnicodeObject *)str1,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6968 (PyUnicodeObject *)str2,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6969 maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6970 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6971 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6972 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6973 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6974 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6975
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6976 PyDoc_STRVAR(replace__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6977 "S.replace (old, new[, maxsplit]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6978 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6979 Return a copy of S with all occurrences of substring\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6980 old replaced by new. If the optional argument maxsplit is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6981 given, only the first maxsplit occurrences are replaced.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6982
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6983 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6984 unicode_replace(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6985 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6986 PyUnicodeObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6987 PyUnicodeObject *str2;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6988 Py_ssize_t maxcount = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6989 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6990
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6991 if (!PyArg_ParseTuple(args, "OO|n:replace", &str1, &str2, &maxcount))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6992 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6993 str1 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6994 if (str1 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6995 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6996 str2 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str2);
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
6997 if (str2 == NULL) {
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
6998 Py_DECREF(str1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6999 return NULL;
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
7000 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7001
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7002 result = replace(self, str1, str2, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7003
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7004 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7005 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7006 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7007 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7008
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7009 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7010 PyObject *unicode_repr(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7011 {
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7012 PyObject *repr;
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7013 Py_UNICODE *p;
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7014 Py_UNICODE *s = PyUnicode_AS_UNICODE(unicode);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7015 Py_ssize_t size = PyUnicode_GET_SIZE(unicode);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7016
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7017 /* XXX(nnorwitz): rather than over-allocating, it would be
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7018 better to choose a different scheme. Perhaps scan the
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7019 first N-chars of the string and allocate based on that size.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7020 */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7021 /* Initial allocation is based on the longest-possible unichr
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7022 escape.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7023
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7024 In wide (UTF-32) builds '\U00xxxxxx' is 10 chars per source
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7025 unichr, so in this case it's the longest unichr escape. In
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7026 narrow (UTF-16) builds this is five chars per source unichr
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7027 since there are two unichrs in the surrogate pair, so in narrow
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7028 (UTF-16) builds it's not the longest unichr escape.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7029
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7030 In wide or narrow builds '\uxxxx' is 6 chars per source unichr,
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7031 so in the narrow (UTF-16) build case it's the longest unichr
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7032 escape.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7033 */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7034
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7035 repr = PyUnicode_FromUnicode(NULL,
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7036 2 /* quotes */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7037 #ifdef Py_UNICODE_WIDE
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7038 + 10*size
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7039 #else
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7040 + 6*size
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7041 #endif
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7042 + 1);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7043 if (repr == NULL)
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7044 return NULL;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7045
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7046 p = PyUnicode_AS_UNICODE(repr);
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7047
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7048 /* Add quote */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7049 *p++ = (findchar(s, size, '\'') &&
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7050 !findchar(s, size, '"')) ? '"' : '\'';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7051 while (size-- > 0) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7052 Py_UNICODE ch = *s++;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7053
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7054 /* Escape quotes and backslashes */
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7055 if ((ch == PyUnicode_AS_UNICODE(repr)[0]) || (ch == '\\')) {
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7056 *p++ = '\\';
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7057 *p++ = ch;
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7058 continue;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7059 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7060
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7061 #ifdef Py_UNICODE_WIDE
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7062 /* Map 21-bit characters to '\U00xxxxxx' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7063 else if (ch >= 0x10000) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7064 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7065 *p++ = 'U';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7066 *p++ = hexdigits[(ch >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7067 *p++ = hexdigits[(ch >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7068 *p++ = hexdigits[(ch >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7069 *p++ = hexdigits[(ch >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7070 *p++ = hexdigits[(ch >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7071 *p++ = hexdigits[(ch >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7072 *p++ = hexdigits[(ch >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7073 *p++ = hexdigits[ch & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7074 continue;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7075 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7076 #else
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7077 /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7078 else if (ch >= 0xD800 && ch < 0xDC00) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7079 Py_UNICODE ch2;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7080 Py_UCS4 ucs;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7081
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7082 ch2 = *s++;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7083 size--;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7084 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7085 ucs = (((ch & 0x03FF) << 10) | (ch2 & 0x03FF)) + 0x00010000;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7086 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7087 *p++ = 'U';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7088 *p++ = hexdigits[(ucs >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7089 *p++ = hexdigits[(ucs >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7090 *p++ = hexdigits[(ucs >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7091 *p++ = hexdigits[(ucs >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7092 *p++ = hexdigits[(ucs >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7093 *p++ = hexdigits[(ucs >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7094 *p++ = hexdigits[(ucs >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7095 *p++ = hexdigits[ucs & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7096 continue;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7097 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7098 /* Fall through: isolated surrogates are copied as-is */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7099 s--;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7100 size++;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7101 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7102 #endif
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7103
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7104 /* Map 16-bit characters to '\uxxxx' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7105 if (ch >= 256) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7106 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7107 *p++ = 'u';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7108 *p++ = hexdigits[(ch >> 12) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7109 *p++ = hexdigits[(ch >> 8) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7110 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7111 *p++ = hexdigits[ch & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7112 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7113
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7114 /* Map special whitespace to '\t', \n', '\r' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7115 else if (ch == '\t') {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7116 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7117 *p++ = 't';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7118 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7119 else if (ch == '\n') {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7120 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7121 *p++ = 'n';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7122 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7123 else if (ch == '\r') {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7124 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7125 *p++ = 'r';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7126 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7127
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7128 /* Map non-printable US ASCII to '\xhh' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7129 else if (ch < ' ' || ch >= 0x7F) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7130 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7131 *p++ = 'x';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7132 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7133 *p++ = hexdigits[ch & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7134 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7135
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7136 /* Copy everything else as-is */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7137 else
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7138 *p++ = (char) ch;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7139 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7140 /* Add quote */
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7141 *p++ = PyUnicode_AS_UNICODE(repr)[0];
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7142
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7143 *p = '\0';
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7144 _PyUnicode_Resize(&repr, p - PyUnicode_AS_UNICODE(repr));
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7145 return repr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7146 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7147
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7148 PyDoc_STRVAR(rfind__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7149 "S.rfind(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7150 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7151 Return the highest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7152 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7153 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7154 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7155 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7156
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7157 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7158 unicode_rfind(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7159 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7160 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7161 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7162 Py_ssize_t end = PY_SSIZE_T_MAX;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7163 Py_ssize_t result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7164
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7165 if (!PyArg_ParseTuple(args, "O|O&O&:rfind", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7166 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7167 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7168 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7169 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7170 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7171
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7172 result = stringlib_rfind_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7173 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7174 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7175 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7176 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7177
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7178 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7179
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7180 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7181 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7182
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7183 PyDoc_STRVAR(rindex__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7184 "S.rindex(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7185 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7186 Like S.rfind() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7187
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7188 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7189 unicode_rindex(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7190 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7191 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7192 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7193 Py_ssize_t end = PY_SSIZE_T_MAX;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7194 Py_ssize_t result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7195
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7196 if (!PyArg_ParseTuple(args, "O|O&O&:rindex", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7197 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7198 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7199 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7200 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7201 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7202
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7203 result = stringlib_rfind_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7204 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7205 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7206 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7207 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7208
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7209 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7210
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7211 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7212 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7213 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7214 }
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7215 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7216 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7217
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7218 PyDoc_STRVAR(rjust__doc__,
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7219 "S.rjust(width[, fillchar]) -> unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7220 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7221 Return S right justified in a Unicode string of length width. Padding is\n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7222 done using the specified fill character (default is a space).");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7223
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7224 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7225 unicode_rjust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7226 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7227 Py_ssize_t width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7228 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7229
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7230 if (!PyArg_ParseTuple(args, "n|O&:rjust", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7231 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7232
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
7233 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7234 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7235 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7236 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7237
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7238 return (PyObject*) pad(self, width - self->length, 0, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7239 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7240
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7241 static PyObject*
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7242 unicode_slice(PyUnicodeObject *self, Py_ssize_t start, Py_ssize_t end)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7243 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7244 /* standard clamping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7245 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7246 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7247 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7248 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7249 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7250 end = self->length;
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
7251 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7252 /* full slice, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7253 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7254 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7255 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7256 if (start > end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7257 start = end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7258 /* copy slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7259 return (PyObject*) PyUnicode_FromUnicode(self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7260 end - start);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7261 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7262
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7263 PyObject *PyUnicode_Split(PyObject *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7264 PyObject *sep,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7265 Py_ssize_t maxsplit)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7266 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7267 PyObject *result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7268
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7269 s = PyUnicode_FromObject(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7270 if (s == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7271 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7272 if (sep != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7273 sep = PyUnicode_FromObject(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7274 if (sep == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7275 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7276 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7277 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7278 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7279
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7280 result = split((PyUnicodeObject *)s, (PyUnicodeObject *)sep, maxsplit);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7281
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7282 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7283 Py_XDECREF(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7284 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7285 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7286
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7287 PyDoc_STRVAR(split__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7288 "S.split([sep [,maxsplit]]) -> list of strings\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7289 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7290 Return a list of the words in S, using sep as the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7291 delimiter string. If maxsplit is given, at most maxsplit\n\
33495
2d6e1854082e Add a missing line continuation character.
Thomas Heller <theller@ctypes.org>
parents: 33487
diff changeset
7292 splits are done. If sep is not specified or is None,\n\
33486
53c1357c9c63 Enhance the docstrings for unicode.split() and string.split()
Walter Dörwald <walter@livinglogic.de>
parents: 33428
diff changeset
7293 any whitespace string is a separator.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7294
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7295 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7296 unicode_split(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7297 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7298 PyObject *substring = Py_None;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7299 Py_ssize_t maxcount = -1;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7300
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7301 if (!PyArg_ParseTuple(args, "|On:split", &substring, &maxcount))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7302 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7303
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7304 if (substring == Py_None)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7305 return split(self, NULL, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7306 else if (PyUnicode_Check(substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7307 return split(self, (PyUnicodeObject *)substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7308 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7309 return PyUnicode_Split((PyObject *)self, substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7310 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7311
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7312 PyObject *
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7313 PyUnicode_Partition(PyObject *str_in, PyObject *sep_in)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7314 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7315 PyObject* str_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7316 PyObject* sep_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7317 PyObject* out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7318
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7319 str_obj = PyUnicode_FromObject(str_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7320 if (!str_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7321 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7322 sep_obj = PyUnicode_FromObject(sep_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7323 if (!sep_obj) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7324 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7325 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7326 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7327
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7328 out = stringlib_partition(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7329 str_obj, PyUnicode_AS_UNICODE(str_obj), PyUnicode_GET_SIZE(str_obj),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7330 sep_obj, PyUnicode_AS_UNICODE(sep_obj), PyUnicode_GET_SIZE(sep_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7331 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7332
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7333 Py_DECREF(sep_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7334 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7335
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7336 return out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7337 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7338
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7339
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7340 PyObject *
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7341 PyUnicode_RPartition(PyObject *str_in, PyObject *sep_in)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7342 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7343 PyObject* str_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7344 PyObject* sep_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7345 PyObject* out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7346
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7347 str_obj = PyUnicode_FromObject(str_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7348 if (!str_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7349 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7350 sep_obj = PyUnicode_FromObject(sep_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7351 if (!sep_obj) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7352 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7353 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7354 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7355
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7356 out = stringlib_rpartition(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7357 str_obj, PyUnicode_AS_UNICODE(str_obj), PyUnicode_GET_SIZE(str_obj),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7358 sep_obj, PyUnicode_AS_UNICODE(sep_obj), PyUnicode_GET_SIZE(sep_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7359 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7360
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7361 Py_DECREF(sep_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7362 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7363
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7364 return out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7365 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7366
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7367 PyDoc_STRVAR(partition__doc__,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7368 "S.partition(sep) -> (head, sep, tail)\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7369 \n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7370 Searches for the separator sep in S, and returns the part before it,\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7371 the separator itself, and the part after it. If the separator is not\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7372 found, returns S and two empty strings.");
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7373
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7374 static PyObject*
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7375 unicode_partition(PyUnicodeObject *self, PyObject *separator)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7376 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7377 return PyUnicode_Partition((PyObject *)self, separator);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7378 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7379
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7380 PyDoc_STRVAR(rpartition__doc__,
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
7381 "S.rpartition(sep) -> (tail, sep, head)\n\
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7382 \n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7383 Searches for the separator sep in S, starting at the end of S, and returns\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7384 the part before it, the separator itself, and the part after it. If the\n\
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
7385 separator is not found, returns two empty strings and S.");
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7386
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7387 static PyObject*
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7388 unicode_rpartition(PyUnicodeObject *self, PyObject *separator)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7389 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7390 return PyUnicode_RPartition((PyObject *)self, separator);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7391 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7392
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7393 PyObject *PyUnicode_RSplit(PyObject *s,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7394 PyObject *sep,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7395 Py_ssize_t maxsplit)
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7396 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7397 PyObject *result;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7398
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7399 s = PyUnicode_FromObject(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7400 if (s == NULL)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7401 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7402 if (sep != NULL) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7403 sep = PyUnicode_FromObject(sep);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7404 if (sep == NULL) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7405 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7406 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7407 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7408 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7409
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7410 result = rsplit((PyUnicodeObject *)s, (PyUnicodeObject *)sep, maxsplit);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7411
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7412 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7413 Py_XDECREF(sep);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7414 return result;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7415 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7416
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7417 PyDoc_STRVAR(rsplit__doc__,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7418 "S.rsplit([sep [,maxsplit]]) -> list of strings\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7419 \n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7420 Return a list of the words in S, using sep as the\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7421 delimiter string, starting at the end of the string and\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7422 working to the front. If maxsplit is given, at most maxsplit\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7423 splits are done. If sep is not specified, any whitespace string\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7424 is a separator.");
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7425
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7426 static PyObject*
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7427 unicode_rsplit(PyUnicodeObject *self, PyObject *args)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7428 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7429 PyObject *substring = Py_None;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7430 Py_ssize_t maxcount = -1;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7431
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7432 if (!PyArg_ParseTuple(args, "|On:rsplit", &substring, &maxcount))
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7433 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7434
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7435 if (substring == Py_None)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7436 return rsplit(self, NULL, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7437 else if (PyUnicode_Check(substring))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7438 return rsplit(self, (PyUnicodeObject *)substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7439 else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7440 return PyUnicode_RSplit((PyObject *)self, substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7441 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7442
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7443 PyDoc_STRVAR(splitlines__doc__,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7444 "S.splitlines([keepends]]) -> list of strings\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7445 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7446 Return a list of the lines in S, breaking at line boundaries.\n\
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7447 Line breaks are not included in the resulting list unless keepends\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7448 is given and true.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7449
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7450 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7451 unicode_splitlines(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7452 {
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7453 int keepends = 0;
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7454
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7455 if (!PyArg_ParseTuple(args, "|i:splitlines", &keepends))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7456 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7457
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7458 return PyUnicode_Splitlines((PyObject *)self, keepends);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7459 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7460
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7461 static
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
7462 PyObject *unicode_str(PyObject *self)
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
7463 {
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7464 if (PyUnicode_CheckExact(self)) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7465 Py_INCREF(self);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7466 return self;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7467 } else
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7468 /* Subtype -- return genuine unicode string with the same value. */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7469 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(self),
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7470 PyUnicode_GET_SIZE(self));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7471 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7472
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7473 PyDoc_STRVAR(swapcase__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7474 "S.swapcase() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7475 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7476 Return a copy of S with uppercase characters converted to lowercase\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7477 and vice versa.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7478
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7479 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7480 unicode_swapcase(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7481 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7482 return fixup(self, fixswapcase);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7483 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7484
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7485 PyDoc_STRVAR(translate__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7486 "S.translate(table) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7487 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7488 Return a copy of the string S, where all characters have been mapped\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7489 through the given translation table, which must be a mapping of\n\
25120
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
7490 Unicode ordinals to Unicode ordinals, Unicode strings or None.\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
7491 Unmapped characters are left untouched. Characters mapped to None\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
7492 are deleted.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7493
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7494 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7495 unicode_translate(PyUnicodeObject *self, PyObject *table)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7496 {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7497 return PyUnicode_TranslateCharmap(self->str,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7498 self->length,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7499 table,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7500 "ignore");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7501 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7502
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7503 PyDoc_STRVAR(upper__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7504 "S.upper() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7505 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7506 Return a copy of S converted to uppercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7507
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7508 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7509 unicode_upper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7510 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7511 return fixup(self, fixupper);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7512 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7513
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7514 PyDoc_STRVAR(zfill__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7515 "S.zfill(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7516 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7517 Pad a numeric string x with zeros on the left, to fill a field\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7518 of the specified width. The string x is never truncated.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7519
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7520 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7521 unicode_zfill(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7522 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7523 Py_ssize_t fill;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7524 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7525
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7526 Py_ssize_t width;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7527 if (!PyArg_ParseTuple(args, "n:zfill", &width))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7528 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7529
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7530 if (self->length >= width) {
23051
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7531 if (PyUnicode_CheckExact(self)) {
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7532 Py_INCREF(self);
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7533 return (PyObject*) self;
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7534 }
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7535 else
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7536 return PyUnicode_FromUnicode(
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7537 PyUnicode_AS_UNICODE(self),
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7538 PyUnicode_GET_SIZE(self)
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7539 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7540 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7541
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7542 fill = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7543
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7544 u = pad(self, fill, 0, '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7545
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7546 if (u == NULL)
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7547 return NULL;
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7548
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7549 if (u->str[fill] == '+' || u->str[fill] == '-') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7550 /* move sign to beginning of string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7551 u->str[0] = u->str[fill];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7552 u->str[fill] = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7553 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7554
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7555 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7556 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7557
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7558 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7559 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7560 unicode_freelistsize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7561 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7562 return PyInt_FromLong(unicode_freelist_size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7563 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7564 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7565
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7566 PyDoc_STRVAR(startswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
7567 "S.startswith(prefix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7568 \n\
28559
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7569 Return True if S starts with the specified prefix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7570 With optional start, test S beginning at that position.\n\
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7571 With optional end, stop comparing S at that position.\n\
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7572 prefix can also be a tuple of strings to try.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7573
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7574 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7575 unicode_startswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7576 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7577 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7578 PyObject *subobj;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7579 PyUnicodeObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7580 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7581 Py_ssize_t end = PY_SSIZE_T_MAX;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7582 int result;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7583
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7584 if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj,
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7585 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7586 return NULL;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7587 if (PyTuple_Check(subobj)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7588 Py_ssize_t i;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7589 for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7590 substring = (PyUnicodeObject *)PyUnicode_FromObject(
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7591 PyTuple_GET_ITEM(subobj, i));
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7592 if (substring == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7593 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7594 result = tailmatch(self, substring, start, end, -1);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7595 Py_DECREF(substring);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7596 if (result) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7597 Py_RETURN_TRUE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7598 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7599 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7600 /* nothing matched */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7601 Py_RETURN_FALSE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7602 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7603 substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7604 if (substring == NULL)
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7605 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7606 result = tailmatch(self, substring, start, end, -1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7607 Py_DECREF(substring);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7608 return PyBool_FromLong(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7609 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7610
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7611
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7612 PyDoc_STRVAR(endswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
7613 "S.endswith(suffix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7614 \n\
28559
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7615 Return True if S ends with the specified suffix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7616 With optional start, test S beginning at that position.\n\
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7617 With optional end, stop comparing S at that position.\n\
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7618 suffix can also be a tuple of strings to try.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7619
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7620 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7621 unicode_endswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7622 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7623 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7624 PyObject *subobj;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7625 PyUnicodeObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7626 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7627 Py_ssize_t end = PY_SSIZE_T_MAX;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7628 int result;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7629
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7630 if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7631 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7632 return NULL;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7633 if (PyTuple_Check(subobj)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7634 Py_ssize_t i;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7635 for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7636 substring = (PyUnicodeObject *)PyUnicode_FromObject(
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7637 PyTuple_GET_ITEM(subobj, i));
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7638 if (substring == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7639 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7640 result = tailmatch(self, substring, start, end, +1);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7641 Py_DECREF(substring);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7642 if (result) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7643 Py_RETURN_TRUE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7644 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7645 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7646 Py_RETURN_FALSE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7647 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7648 substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7649 if (substring == NULL)
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7650 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7651
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7652 result = tailmatch(self, substring, start, end, +1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7653 Py_DECREF(substring);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7654 return PyBool_FromLong(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7655 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7656
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7657
27521
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7658
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7659 static PyObject *
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7660 unicode_getnewargs(PyUnicodeObject *v)
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7661 {
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7662 return Py_BuildValue("(u#)", v->str, v->length);
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7663 }
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7664
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7665
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7666 static PyMethodDef unicode_methods[] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7667
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7668 /* Order is according to common usage: often used methods should
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7669 appear first, since lookup is done sequentially. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7670
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7671 {"encode", (PyCFunction) unicode_encode, METH_VARARGS, encode__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7672 {"replace", (PyCFunction) unicode_replace, METH_VARARGS, replace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7673 {"split", (PyCFunction) unicode_split, METH_VARARGS, split__doc__},
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7674 {"rsplit", (PyCFunction) unicode_rsplit, METH_VARARGS, rsplit__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7675 {"join", (PyCFunction) unicode_join, METH_O, join__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7676 {"capitalize", (PyCFunction) unicode_capitalize, METH_NOARGS, capitalize__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7677 {"title", (PyCFunction) unicode_title, METH_NOARGS, title__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7678 {"center", (PyCFunction) unicode_center, METH_VARARGS, center__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7679 {"count", (PyCFunction) unicode_count, METH_VARARGS, count__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7680 {"expandtabs", (PyCFunction) unicode_expandtabs, METH_VARARGS, expandtabs__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7681 {"find", (PyCFunction) unicode_find, METH_VARARGS, find__doc__},
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7682 {"partition", (PyCFunction) unicode_partition, METH_O, partition__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7683 {"index", (PyCFunction) unicode_index, METH_VARARGS, index__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7684 {"ljust", (PyCFunction) unicode_ljust, METH_VARARGS, ljust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7685 {"lower", (PyCFunction) unicode_lower, METH_NOARGS, lower__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
7686 {"lstrip", (PyCFunction) unicode_lstrip, METH_VARARGS, lstrip__doc__},
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
7687 {"decode", (PyCFunction) unicode_decode, METH_VARARGS, decode__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7688 /* {"maketrans", (PyCFunction) unicode_maketrans, METH_VARARGS, maketrans__doc__}, */
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7689 {"rfind", (PyCFunction) unicode_rfind, METH_VARARGS, rfind__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7690 {"rindex", (PyCFunction) unicode_rindex, METH_VARARGS, rindex__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7691 {"rjust", (PyCFunction) unicode_rjust, METH_VARARGS, rjust__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
7692 {"rstrip", (PyCFunction) unicode_rstrip, METH_VARARGS, rstrip__doc__},
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7693 {"rpartition", (PyCFunction) unicode_rpartition, METH_O, rpartition__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7694 {"splitlines", (PyCFunction) unicode_splitlines, METH_VARARGS, splitlines__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
7695 {"strip", (PyCFunction) unicode_strip, METH_VARARGS, strip__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7696 {"swapcase", (PyCFunction) unicode_swapcase, METH_NOARGS, swapcase__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7697 {"translate", (PyCFunction) unicode_translate, METH_O, translate__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7698 {"upper", (PyCFunction) unicode_upper, METH_NOARGS, upper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7699 {"startswith", (PyCFunction) unicode_startswith, METH_VARARGS, startswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7700 {"endswith", (PyCFunction) unicode_endswith, METH_VARARGS, endswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7701 {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7702 {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7703 {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7704 {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7705 {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7706 {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7707 {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7708 {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7709 {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7710 {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7711 #if 0
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7712 {"capwords", (PyCFunction) unicode_capwords, METH_NOARGS, capwords__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7713 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7714
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7715 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7716 /* This one is just used for debugging the implementation. */
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7717 {"freelistsize", (PyCFunction) unicode_freelistsize, METH_NOARGS},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7718 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7719
27521
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7720 {"__getnewargs__", (PyCFunction)unicode_getnewargs, METH_NOARGS},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7721 {NULL, NULL}
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7722 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7723
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7724 static PyObject *
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7725 unicode_mod(PyObject *v, PyObject *w)
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7726 {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7727 if (!PyUnicode_Check(v)) {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7728 Py_INCREF(Py_NotImplemented);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7729 return Py_NotImplemented;
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7730 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7731 return PyUnicode_Format(v, w);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7732 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7733
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7734 static PyNumberMethods unicode_as_number = {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7735 0, /*nb_add*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7736 0, /*nb_subtract*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7737 0, /*nb_multiply*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7738 unicode_mod, /*nb_remainder*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7739 };
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7740
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7741 static PySequenceMethods unicode_as_sequence = {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7742 (lenfunc) unicode_length, /* sq_length */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7743 PyUnicode_Concat, /* sq_concat */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7744 (ssizeargfunc) unicode_repeat, /* sq_repeat */
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7745 (ssizeargfunc) unicode_getitem, /* sq_item */
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7746 (ssizessizeargfunc) unicode_slice, /* sq_slice */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7747 0, /* sq_ass_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7748 0, /* sq_ass_slice */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7749 PyUnicode_Contains, /* sq_contains */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7750 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7751
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7752 static PyObject*
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7753 unicode_subscript(PyUnicodeObject* self, PyObject* item)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7754 {
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
7755 if (PyIndex_Check(item)) {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
7756 Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7757 if (i == -1 && PyErr_Occurred())
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7758 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7759 if (i < 0)
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7760 i += PyUnicode_GET_SIZE(self);
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7761 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7762 } else if (PySlice_Check(item)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7763 Py_ssize_t start, stop, step, slicelength, cur, i;
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7764 Py_UNICODE* source_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7765 Py_UNICODE* result_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7766 PyObject* result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7767
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7768 if (PySlice_GetIndicesEx((PySliceObject*)item, PyUnicode_GET_SIZE(self),
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7769 &start, &stop, &step, &slicelength) < 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7770 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7771 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7772
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7773 if (slicelength <= 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7774 return PyUnicode_FromUnicode(NULL, 0);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7775 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7776 source_buf = PyUnicode_AS_UNICODE((PyObject*)self);
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7777 result_buf = (Py_UNICODE *)PyMem_MALLOC(slicelength*
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7778 sizeof(Py_UNICODE));
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7779
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7780 if (result_buf == NULL)
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7781 return PyErr_NoMemory();
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7782
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7783 for (cur = start, i = 0; i < slicelength; cur += step, i++) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7784 result_buf[i] = source_buf[cur];
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7785 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7786
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7787 result = PyUnicode_FromUnicode(result_buf, slicelength);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7788 PyMem_FREE(result_buf);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7789 return result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7790 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7791 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7792 PyErr_SetString(PyExc_TypeError, "string indices must be integers");
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7793 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7794 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7795 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7796
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7797 static PyMappingMethods unicode_as_mapping = {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7798 (lenfunc)unicode_length, /* mp_length */
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7799 (binaryfunc)unicode_subscript, /* mp_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7800 (objobjargproc)0, /* mp_ass_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7801 };
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7802
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7803 static Py_ssize_t
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7804 unicode_buffer_getreadbuf(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7805 Py_ssize_t index,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7806 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7807 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7808 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7809 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7810 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7811 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7812 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7813 *ptr = (void *) self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7814 return PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7815 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7816
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7817 static Py_ssize_t
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7818 unicode_buffer_getwritebuf(PyUnicodeObject *self, Py_ssize_t index,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7819 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7820 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7821 PyErr_SetString(PyExc_TypeError,
23837
359a57e96d5e Fix typo in exception message
Neal Norwitz <nnorwitz@gmail.com>
parents: 23832
diff changeset
7822 "cannot use unicode as modifiable buffer");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7823 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7824 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7825
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7826 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7827 unicode_buffer_getsegcount(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7828 Py_ssize_t *lenp)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7829 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7830 if (lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7831 *lenp = PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7832 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7833 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7834
36519
d6d415df0df0 Use Py_ssize_t for counts and sizes.
Martin v. Löwis <martin@v.loewis.de>
parents: 36510
diff changeset
7835 static Py_ssize_t
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7836 unicode_buffer_getcharbuf(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7837 Py_ssize_t index,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7838 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7839 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7840 PyObject *str;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7841
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7842 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7843 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7844 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7845 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7846 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
7847 str = _PyUnicode_AsDefaultEncodedString((PyObject *)self, NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7848 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7849 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7850 *ptr = (void *) PyString_AS_STRING(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7851 return PyString_GET_SIZE(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7852 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7853
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7854 /* Helpers for PyUnicode_Format() */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7855
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7856 static PyObject *
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7857 getnextarg(PyObject *args, Py_ssize_t arglen, Py_ssize_t *p_argidx)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7858 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7859 Py_ssize_t argidx = *p_argidx;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7860 if (argidx < arglen) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7861 (*p_argidx)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7862 if (arglen < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7863 return args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7864 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7865 return PyTuple_GetItem(args, argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7866 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7867 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7868 "not enough arguments for format string");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7869 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7870 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7871
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7872 #define F_LJUST (1<<0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7873 #define F_SIGN (1<<1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7874 #define F_BLANK (1<<2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7875 #define F_ALT (1<<3)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7876 #define F_ZERO (1<<4)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7877
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7878 static Py_ssize_t
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7879 strtounicode(Py_UNICODE *buffer, const char *charbuffer)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7880 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7881 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7882 Py_ssize_t len = strlen(charbuffer);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7883 for (i = len - 1; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7884 buffer[i] = (Py_UNICODE) charbuffer[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7885
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7886 return len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7887 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7888
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7889 static int
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7890 doubletounicode(Py_UNICODE *buffer, size_t len, const char *format, double x)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7891 {
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7892 Py_ssize_t result;
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7893
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7894 PyOS_ascii_formatd((char *)buffer, len, format, x);
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7895 result = strtounicode(buffer, (char *)buffer);
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7896 return Py_SAFE_DOWNCAST(result, Py_ssize_t, int);
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7897 }
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7898
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7899 static int
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7900 longtounicode(Py_UNICODE *buffer, size_t len, const char *format, long x)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7901 {
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7902 Py_ssize_t result;
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7903
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7904 PyOS_snprintf((char *)buffer, len, format, x);
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7905 result = strtounicode(buffer, (char *)buffer);
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7906 return Py_SAFE_DOWNCAST(result, Py_ssize_t, int);
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7907 }
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7908
24736
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7909 /* XXX To save some code duplication, formatfloat/long/int could have been
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7910 shared with stringobject.c, converting from 8-bit to Unicode after the
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7911 formatting is done. */
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7912
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7913 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7914 formatfloat(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7915 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7916 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7917 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7918 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7919 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7920 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7921 /* fmt = '%#.' + `prec` + `type`
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7922 worst case length = 3 + 10 (len of INT_MAX) + 1 = 14 (use 20)*/
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7923 char fmt[20];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7924 double x;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7925
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7926 x = PyFloat_AsDouble(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7927 if (x == -1.0 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7928 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7929 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7930 prec = 6;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7931 if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7932 type = 'g';
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7933 /* Worst case length calc to ensure no buffer overrun:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7934
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7935 'g' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7936 fmt = %#.<prec>g
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7937 buf = '-' + [0-9]*prec + '.' + 'e+' + (longest exp
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7938 for any double rep.)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7939 len = 1 + prec + 1 + 2 + 5 = 9 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7940
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7941 'f' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7942 buf = '-' + [0-9]*x + '.' + [0-9]*prec (with x < 50)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7943 len = 1 + 50 + 1 + prec = 52 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7944
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7945 If prec=0 the effective precision is 1 (the leading digit is
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7946 always given), therefore increase the length by one.
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7947
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7948 */
42309
bd6acfdb47d2 Merged revisions 56301-56442 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42291
diff changeset
7949 if (((type == 'g' || type == 'G') &&
bd6acfdb47d2 Merged revisions 56301-56442 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42291
diff changeset
7950 buflen <= (size_t)10 + (size_t)prec) ||
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7951 (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) {
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7952 PyErr_SetString(PyExc_OverflowError,
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7953 "formatted float is too long (precision too large?)");
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7954 return -1;
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7955 }
21082
c8d775b9ff92 formatfloat(), formatint(): Conversion of sprintf() to PyOS_snprintf()
Barry Warsaw <barry@python.org>
parents: 21067
diff changeset
7956 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%d%c",
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7957 (flags&F_ALT) ? "#" : "",
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7958 prec, type);
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7959 return doubletounicode(buf, buflen, fmt, x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7960 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7961
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7962 static PyObject*
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7963 formatlong(PyObject *val, int flags, int prec, int type)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7964 {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7965 char *buf;
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7966 int len;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7967 PyObject *str; /* temporary string object. */
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7968 PyObject *result;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7969
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7970 str = _PyString_FormatLong(val, flags, prec, type, &buf, &len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7971 if (!str)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7972 return NULL;
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7973 result = PyUnicode_FromStringAndSize(buf, len);
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7974 Py_DECREF(str);
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7975 return result;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7976 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7977
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7978 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7979 formatint(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7980 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7981 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7982 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7983 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7984 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7985 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7986 /* fmt = '%#.' + `prec` + 'l' + `type`
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7987 * worst case length = 3 + 19 (worst len of INT_MAX on 64-bit machine)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7988 * + 1 + 1
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7989 * = 24
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7990 */
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7991 char fmt[64]; /* plenty big enough! */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7992 char *sign;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7993 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7994
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7995 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7996 if (x == -1 && PyErr_Occurred())
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7997 return -1;
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7998 if (x < 0 && type == 'u') {
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7999 type = 'd';
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8000 }
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8001 if (x < 0 && (type == 'x' || type == 'X' || type == 'o'))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8002 sign = "-";
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8003 else
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8004 sign = "";
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8005 if (prec < 0)
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8006 prec = 1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8007
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8008 /* buf = '+'/'-'/'' + '0'/'0x'/'' + '[0-9]'*max(prec, len(x in octal))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8009 * worst case buf = '-0x' + [0-9]*prec, where prec >= 11
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8010 */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8011 if (buflen <= 14 || buflen <= (size_t)3 + (size_t)prec) {
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8012 PyErr_SetString(PyExc_OverflowError,
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8013 "formatted integer is too long (precision too large?)");
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8014 return -1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8015 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8016
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8017 if ((flags & F_ALT) &&
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8018 (type == 'x' || type == 'X' || type == 'o')) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8019 /* When converting under %#o, %#x or %#X, there are a number
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8020 * of issues that cause pain:
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8021 * - for %#o, we want a different base marker than C
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8022 * - when 0 is being converted, the C standard leaves off
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8023 * the '0x' or '0X', which is inconsistent with other
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8024 * %#x/%#X conversions and inconsistent with Python's
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8025 * hex() function
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8026 * - there are platforms that violate the standard and
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8027 * convert 0 with the '0x' or '0X'
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8028 * (Metrowerks, Compaq Tru64)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8029 * - there are platforms that give '0x' when converting
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8030 * under %#X, but convert 0 in accordance with the
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8031 * standard (OS/2 EMX)
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8032 *
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8033 * We can achieve the desired consistency by inserting our
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8034 * own '0x' or '0X' prefix, and substituting %x/%X in place
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8035 * of %#x/%#X.
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8036 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8037 * Note that this is the same approach as used in
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8038 * formatint() in stringobject.c
22053
49197fd76362 OS/2 EMX port changes (Objects part of patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 21886
diff changeset
8039 */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8040 PyOS_snprintf(fmt, sizeof(fmt), "%s0%c%%.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8041 sign, type, prec, type);
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8042 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8043 else {
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8044 PyOS_snprintf(fmt, sizeof(fmt), "%s%%%s.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8045 sign, (flags&F_ALT) ? "#" : "",
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8046 prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8047 }
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8048 if (sign[0])
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
8049 return longtounicode(buf, buflen, fmt, -x);
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8050 else
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
8051 return longtounicode(buf, buflen, fmt, x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8052 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8053
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8054 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8055 formatchar(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8056 size_t buflen,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8057 PyObject *v)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8058 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8059 /* presume that the buffer is at least 2 characters long */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8060 if (PyUnicode_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8061 if (PyUnicode_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8062 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8063 buf[0] = PyUnicode_AS_UNICODE(v)[0];
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8064 }
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8065
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8066 else if (PyString_Check(v)) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8067 if (PyString_GET_SIZE(v) != 1)
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8068 goto onError;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8069 buf[0] = (Py_UNICODE)PyString_AS_STRING(v)[0];
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8070 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8071
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8072 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8073 /* Integer input truncated to a character */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8074 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8075 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8076 if (x == -1 && PyErr_Occurred())
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8077 goto onError;
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8078 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8079 if (x < 0 || x > 0x10ffff) {
28494
32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises
Walter Dörwald <walter@livinglogic.de>
parents: 28242
diff changeset
8080 PyErr_SetString(PyExc_OverflowError,
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8081 "%c arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8082 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8083 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8084 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8085 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8086 if (x < 0 || x > 0xffff) {
28494
32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises
Walter Dörwald <walter@livinglogic.de>
parents: 28242
diff changeset
8087 PyErr_SetString(PyExc_OverflowError,
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8088 "%c arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8089 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8090 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8091 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8092 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8093 buf[0] = (Py_UNICODE) x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8094 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8095 buf[1] = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8096 return 1;
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8097
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8098 onError:
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8099 PyErr_SetString(PyExc_TypeError,
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8100 "%c requires int or char");
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8101 return -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8102 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8103
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8104 /* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8105
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8106 FORMATBUFLEN is the length of the buffer in which the floats, ints, &
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8107 chars are formatted. XXX This is a magic number. Each formatting
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8108 routine does bounds checking to ensure no overflow, but a better
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8109 solution may be to malloc a buffer of appropriate size for each
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8110 format. For now, the current solution is sufficient.
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8111 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8112 #define FORMATBUFLEN (size_t)120
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8113
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8114 PyObject *PyUnicode_Format(PyObject *format,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8115 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8116 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8117 Py_UNICODE *fmt, *res;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8118 Py_ssize_t fmtcnt, rescnt, reslen, arglen, argidx;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8119 int args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8120 PyUnicodeObject *result = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8121 PyObject *dict = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8122 PyObject *uformat;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8123
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8124 if (format == NULL || args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8125 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8126 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8127 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8128 uformat = PyUnicode_FromObject(format);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8129 if (uformat == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8130 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8131 fmt = PyUnicode_AS_UNICODE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8132 fmtcnt = PyUnicode_GET_SIZE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8133
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8134 reslen = rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8135 result = _PyUnicode_New(reslen);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8136 if (result == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8137 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8138 res = PyUnicode_AS_UNICODE(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8139
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8140 if (PyTuple_Check(args)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8141 arglen = PyTuple_Size(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8142 argidx = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8143 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8144 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8145 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8146 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8147 }
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
8148 if (Py_Type(args)->tp_as_mapping && !PyTuple_Check(args) &&
26164
63a08639dae2 Fix SF # 635969, No error "not all arguments converted"
Neal Norwitz <nnorwitz@gmail.com>
parents: 25895
diff changeset
8149 !PyObject_TypeCheck(args, &PyBaseString_Type))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8150 dict = args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8151
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8152 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8153 if (*fmt != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8154 if (--rescnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8155 rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8156 reslen += rescnt;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8157 if (_PyUnicode_Resize(&result, reslen) < 0)
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
8158 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8159 res = PyUnicode_AS_UNICODE(result) + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8160 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8161 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8162 *res++ = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8163 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8164 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8165 /* Got a format specifier */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8166 int flags = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8167 Py_ssize_t width = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8168 int prec = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8169 Py_UNICODE c = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8170 Py_UNICODE fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8171 PyObject *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8172 PyObject *temp = NULL;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8173 Py_UNICODE *pbuf;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8174 Py_UNICODE sign;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8175 Py_ssize_t len;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8176 Py_UNICODE formatbuf[FORMATBUFLEN]; /* For format{float,int,char}() */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8177
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8178 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8179 if (*fmt == '(') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8180 Py_UNICODE *keystart;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8181 Py_ssize_t keylen;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8182 PyObject *key;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8183 int pcount = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8184
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8185 if (dict == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8186 PyErr_SetString(PyExc_TypeError,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8187 "format requires a mapping");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8188 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8189 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8190 ++fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8191 --fmtcnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8192 keystart = fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8193 /* Skip over balanced parentheses */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8194 while (pcount > 0 && --fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8195 if (*fmt == ')')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8196 --pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8197 else if (*fmt == '(')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8198 ++pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8199 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8200 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8201 keylen = fmt - keystart - 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8202 if (fmtcnt < 0 || pcount > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8203 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8204 "incomplete format key");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8205 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8206 }
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8207 #if 0
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8208 /* keys are converted to strings using UTF-8 and
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8209 then looked up since Python uses strings to hold
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8210 variables names etc. in its namespaces and we
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8211 wouldn't want to break common idioms. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8212 key = PyUnicode_EncodeUTF8(keystart,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8213 keylen,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8214 NULL);
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8215 #else
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8216 key = PyUnicode_FromUnicode(keystart, keylen);
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8217 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8218 if (key == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8219 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8220 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8221 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8222 args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8223 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8224 args = PyObject_GetItem(dict, key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8225 Py_DECREF(key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8226 if (args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8227 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8228 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8229 args_owned = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8230 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8231 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8232 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8233 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8234 switch (c = *fmt++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8235 case '-': flags |= F_LJUST; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8236 case '+': flags |= F_SIGN; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8237 case ' ': flags |= F_BLANK; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8238 case '#': flags |= F_ALT; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8239 case '0': flags |= F_ZERO; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8240 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8241 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8242 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8243 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8244 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8245 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8246 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8247 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8248 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8249 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8250 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8251 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8252 width = PyInt_AsLong(v);
40626
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8253 if (width == -1 && PyErr_Occurred())
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8254 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8255 if (width < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8256 flags |= F_LJUST;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8257 width = -width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8258 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8259 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8260 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8261 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8262 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8263 width = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8264 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8265 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8266 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8267 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8268 if ((width*10) / 10 != width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8269 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8270 "width too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8271 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8272 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8273 width = width*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8274 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8275 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8276 if (c == '.') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8277 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8278 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8279 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8280 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8281 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8282 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8283 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8284 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8285 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8286 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8287 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8288 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8289 prec = PyInt_AsLong(v);
40626
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8290 if (prec == -1 && PyErr_Occurred())
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8291 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8292 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8293 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8294 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8295 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8296 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8297 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8298 prec = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8299 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8300 c = Py_CHARMASK(*fmt++);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8301 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8302 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8303 if ((prec*10) / 10 != prec) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8304 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8305 "prec too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8306 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8307 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8308 prec = prec*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8309 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8310 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8311 } /* prec */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8312 if (fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8313 if (c == 'h' || c == 'l' || c == 'L') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8314 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8315 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8316 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8317 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8318 if (fmtcnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8319 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8320 "incomplete format");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8321 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8322 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8323 if (c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8324 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8325 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8326 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8327 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8328 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8329 fill = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8330 switch (c) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8331
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8332 case '%':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8333 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8334 /* presume that buffer length is at least 1 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8335 pbuf[0] = '%';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8336 len = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8337 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8338
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8339 case 's':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8340 case 'r':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8341 if (PyUnicode_Check(v) && c == 's') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8342 temp = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8343 Py_INCREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8344 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8345 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8346 PyObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8347 if (c == 's')
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8348 temp = PyObject_Unicode(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8349 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8350 temp = PyObject_Repr(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8351 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8352 goto onError;
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8353 if (PyUnicode_Check(temp))
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8354 /* nothing to do */;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8355 else if (PyString_Check(temp)) {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8356 /* convert to string to Unicode */
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8357 unicode = PyUnicode_Decode(PyString_AS_STRING(temp),
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8358 PyString_GET_SIZE(temp),
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8359 NULL,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8360 "strict");
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8361 Py_DECREF(temp);
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8362 temp = unicode;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8363 if (temp == NULL)
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8364 goto onError;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8365 }
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8366 else {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8367 Py_DECREF(temp);
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8368 PyErr_SetString(PyExc_TypeError,
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8369 "%s argument has non-string str()");
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8370 goto onError;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8371 }
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8372 }
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8373 pbuf = PyUnicode_AS_UNICODE(temp);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8374 len = PyUnicode_GET_SIZE(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8375 if (prec >= 0 && len > prec)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8376 len = prec;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8377 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8378
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8379 case 'i':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8380 case 'd':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8381 case 'u':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8382 case 'o':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8383 case 'x':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8384 case 'X':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8385 if (c == 'i')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8386 c = 'd';
15455
d0146d128897 Fox for SF bug #123859: %[duxXo] long formats inconsistent.
Tim Peters <tim.peters@gmail.com>
parents: 15001
diff changeset
8387 if (PyLong_Check(v)) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8388 temp = formatlong(v, flags, prec, c);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8389 if (!temp)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8390 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8391 pbuf = PyUnicode_AS_UNICODE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8392 len = PyUnicode_GET_SIZE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8393 sign = 1;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8394 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8395 else {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8396 pbuf = formatbuf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8397 len = formatint(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8398 flags, prec, c, v);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8399 if (len < 0)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8400 goto onError;
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8401 sign = 1;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8402 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8403 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8404 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8405 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8406
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8407 case 'e':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8408 case 'E':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8409 case 'f':
30216
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
8410 case 'F':
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8411 case 'g':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8412 case 'G':
30216
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
8413 if (c == 'F')
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
8414 c = 'f';
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8415 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8416 len = formatfloat(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8417 flags, prec, c, v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8418 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8419 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8420 sign = 1;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8421 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8422 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8423 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8424
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8425 case 'c':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8426 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8427 len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8428 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8429 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8430 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8431
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8432 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8433 PyErr_Format(PyExc_ValueError,
15579
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the
Andrew M. Kuchling <amk@amk.ca>
parents: 15455
diff changeset
8434 "unsupported format character '%c' (0x%x) "
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
8435 "at index %zd",
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8436 (31<=c && c<=126) ? (char)c : '?',
25375
4df0e691f4da Add cast to avoid compiler warning.
Marc-André Lemburg <mal@egenix.com>
parents: 25248
diff changeset
8437 (int)c,
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
8438 (Py_ssize_t)(fmt - 1 -
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
8439 PyUnicode_AS_UNICODE(uformat)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8440 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8441 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8442 if (sign) {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8443 if (*pbuf == '-' || *pbuf == '+') {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8444 sign = *pbuf++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8445 len--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8446 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8447 else if (flags & F_SIGN)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8448 sign = '+';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8449 else if (flags & F_BLANK)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8450 sign = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8451 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8452 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8453 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8454 if (width < len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8455 width = len;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
8456 if (rescnt - (sign != 0) < width) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8457 reslen -= rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8458 rescnt = width + fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8459 reslen += rescnt;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
8460 if (reslen < 0) {
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
8461 Py_XDECREF(temp);
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8462 PyErr_NoMemory();
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8463 goto onError;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
8464 }
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8465 if (_PyUnicode_Resize(&result, reslen) < 0) {
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8466 Py_XDECREF(temp);
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8467 goto onError;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8468 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8469 res = PyUnicode_AS_UNICODE(result)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8470 + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8471 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8472 if (sign) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8473 if (fill != ' ')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8474 *res++ = sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8475 rescnt--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8476 if (width > len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8477 width--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8478 }
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8479 if ((flags & F_ALT) && (c == 'x' || c == 'X' || c == 'o')) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8480 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8481 assert(pbuf[1] == c);
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8482 if (fill != ' ') {
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8483 *res++ = *pbuf++;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8484 *res++ = *pbuf++;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8485 }
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8486 rescnt -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8487 width -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8488 if (width < 0)
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8489 width = 0;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8490 len -= 2;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8491 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8492 if (width > len && !(flags & F_LJUST)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8493 do {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8494 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8495 *res++ = fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8496 } while (--width > len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8497 }
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8498 if (fill == ' ') {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8499 if (sign)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8500 *res++ = sign;
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8501 if ((flags & F_ALT) && (c == 'x' || c == 'X' || c == 'o')) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8502 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8503 assert(pbuf[1] == c);
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8504 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8505 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8506 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8507 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8508 Py_UNICODE_COPY(res, pbuf, len);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8509 res += len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8510 rescnt -= len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8511 while (--width >= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8512 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8513 *res++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8514 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8515 if (dict && (argidx < arglen) && c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8516 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
8517 "not all arguments converted during string formatting");
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8518 Py_XDECREF(temp);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8519 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8520 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8521 Py_XDECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8522 } /* '%' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8523 } /* until end */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8524 if (argidx < arglen && !dict) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8525 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
8526 "not all arguments converted during string formatting");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8527 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8528 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8529
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8530 if (_PyUnicode_Resize(&result, reslen - rescnt) < 0)
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8531 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8532 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8533 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8534 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8535 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8536 return (PyObject *)result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8537
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8538 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8539 Py_XDECREF(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8540 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8541 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8542 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8543 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8544 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8545 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8546
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8547 static PyBufferProcs unicode_as_buffer = {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8548 (readbufferproc) unicode_buffer_getreadbuf,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8549 (writebufferproc) unicode_buffer_getwritebuf,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8550 (segcountproc) unicode_buffer_getsegcount,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8551 (charbufferproc) unicode_buffer_getcharbuf,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8552 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8553
24280
1598a1e473e2 staticforward bites the dust.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24071
diff changeset
8554 static PyObject *
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8555 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8556
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8557 static PyObject *
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8558 unicode_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8559 {
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8560 PyObject *x = NULL;
42184
bf5d04e0da4c Make test_descr.py pass. Had to disable a few tests, remove references
Guido van Rossum <guido@python.org>
parents: 42123
diff changeset
8561 static char *kwlist[] = {"object", "encoding", "errors", 0};
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8562 char *encoding = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8563 char *errors = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8564
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8565 if (type != &PyUnicode_Type)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8566 return unicode_subtype_new(type, args, kwds);
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8567 if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:unicode",
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8568 kwlist, &x, &encoding, &errors))
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8569 return NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8570 if (x == NULL)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8571 return (PyObject *)_PyUnicode_New(0);
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
8572 if (encoding == NULL && errors == NULL)
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
8573 return PyObject_Unicode(x);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
8574 else
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8575 return PyUnicode_FromEncodedObject(x, encoding, errors);
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8576 }
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8577
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8578 static PyObject *
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8579 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8580 {
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8581 PyUnicodeObject *tmp, *pnew;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8582 Py_ssize_t n;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8583
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8584 assert(PyType_IsSubtype(type, &PyUnicode_Type));
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8585 tmp = (PyUnicodeObject *)unicode_new(&PyUnicode_Type, args, kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8586 if (tmp == NULL)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8587 return NULL;
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8588 assert(PyUnicode_Check(tmp));
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8589 pnew = (PyUnicodeObject *) type->tp_alloc(type, n = tmp->length);
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8590 if (pnew == NULL) {
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8591 Py_DECREF(tmp);
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8592 return NULL;
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8593 }
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8594 pnew->str = PyMem_NEW(Py_UNICODE, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8595 if (pnew->str == NULL) {
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8596 _Py_ForgetReference((PyObject *)pnew);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
8597 PyObject_Del(pnew);
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8598 Py_DECREF(tmp);
27826
7dffcc498c45 Add more missing PyErr_NoMemory() after failled memory allocs
Neal Norwitz <nnorwitz@gmail.com>
parents: 27774
diff changeset
8599 return PyErr_NoMemory();
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8600 }
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8601 Py_UNICODE_COPY(pnew->str, tmp->str, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8602 pnew->length = n;
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8603 pnew->hash = tmp->hash;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8604 Py_DECREF(tmp);
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8605 return (PyObject *)pnew;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8606 }
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8607
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
8608 PyDoc_STRVAR(unicode_doc,
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8609 "unicode(string [, encoding[, errors]]) -> object\n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8610 \n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8611 Create a new Unicode object from the given encoded string.\n\
24392
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
8612 encoding defaults to the current default string encoding.\n\
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
8613 errors can be 'strict', 'replace' or 'ignore' and defaults to 'strict'.");
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8614
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8615 static PyObject *unicode_iter(PyObject *seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8616
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8617 PyTypeObject PyUnicode_Type = {
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
8618 PyVarObject_HEAD_INIT(&PyType_Type, 0)
41585
825d966338ea Rename 'unicode' to 'str' in its tp_name field. Rename 'str' to 'str8'.
Guido van Rossum <guido@python.org>
parents: 41558
diff changeset
8619 "str", /* tp_name */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8620 sizeof(PyUnicodeObject), /* tp_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8621 0, /* tp_itemsize */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8622 /* Slots */
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
8623 (destructor)unicode_dealloc, /* tp_dealloc */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8624 0, /* tp_print */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8625 0, /* tp_getattr */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8626 0, /* tp_setattr */
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
8627 0, /* tp_compare */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
8628 unicode_repr, /* tp_repr */
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
8629 &unicode_as_number, /* tp_as_number */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8630 &unicode_as_sequence, /* tp_as_sequence */
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
8631 &unicode_as_mapping, /* tp_as_mapping */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8632 (hashfunc) unicode_hash, /* tp_hash*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8633 0, /* tp_call*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8634 (reprfunc) unicode_str, /* tp_str */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8635 PyObject_GenericGetAttr, /* tp_getattro */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8636 0, /* tp_setattro */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8637 &unicode_as_buffer, /* tp_as_buffer */
40909
f2f0b41f270d Merged revisions 53875-53911 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 40626
diff changeset
8638 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE |
f2f0b41f270d Merged revisions 53875-53911 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 40626
diff changeset
8639 Py_TPFLAGS_UNICODE_SUBCLASS, /* tp_flags */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8640 unicode_doc, /* tp_doc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8641 0, /* tp_traverse */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8642 0, /* tp_clear */
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
8643 PyUnicode_RichCompare, /* tp_richcompare */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8644 0, /* tp_weaklistoffset */
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8645 unicode_iter, /* tp_iter */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8646 0, /* tp_iternext */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8647 unicode_methods, /* tp_methods */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8648 0, /* tp_members */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8649 0, /* tp_getset */
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
8650 &PyBaseString_Type, /* tp_base */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8651 0, /* tp_dict */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8652 0, /* tp_descr_get */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8653 0, /* tp_descr_set */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8654 0, /* tp_dictoffset */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8655 0, /* tp_init */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8656 0, /* tp_alloc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8657 unicode_new, /* tp_new */
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
8658 PyObject_Del, /* tp_free */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8659 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8660
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8661 /* Initialize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8662
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
8663 void _PyUnicode_Init(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8664 {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8665 int i;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8666
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8667 /* XXX - move this array to unicodectype.c ? */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8668 Py_UNICODE linebreak[] = {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8669 0x000A, /* LINE FEED */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8670 0x000D, /* CARRIAGE RETURN */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8671 0x001C, /* FILE SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8672 0x001D, /* GROUP SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8673 0x001E, /* RECORD SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8674 0x0085, /* NEXT LINE */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8675 0x2028, /* LINE SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8676 0x2029, /* PARAGRAPH SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8677 };
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8678
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8679 /* Init the implementation */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8680 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8681 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8682 unicode_empty = _PyUnicode_New(0);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8683 if (!unicode_empty)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8684 return;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8685
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8686 for (i = 0; i < 256; i++)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8687 unicode_latin1[i] = NULL;
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
8688 if (PyType_Ready(&PyUnicode_Type) < 0)
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
8689 Py_FatalError("Can't initialize 'unicode'");
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8690
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8691 /* initialize the linebreak bloom filter */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8692 bloom_linebreak = make_bloom_mask(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8693 linebreak, sizeof(linebreak) / sizeof(linebreak[0])
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8694 );
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8695
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8696 PyType_Ready(&EncodingMapType);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8697 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8698
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8699 /* Finalize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8700
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8701 void
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
8702 _PyUnicode_Fini(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8703 {
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
8704 PyUnicodeObject *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8705 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8706
14998
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
8707 Py_XDECREF(unicode_empty);
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
8708 unicode_empty = NULL;
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
8709
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8710 for (i = 0; i < 256; i++) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8711 if (unicode_latin1[i]) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8712 Py_DECREF(unicode_latin1[i]);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8713 unicode_latin1[i] = NULL;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8714 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8715 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8716
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
8717 for (u = unicode_freelist; u != NULL;) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8718 PyUnicodeObject *v = u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8719 u = *(PyUnicodeObject **)u;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
8720 if (v->str)
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
8721 PyMem_DEL(v->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
8722 Py_XDECREF(v->defenc);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
8723 PyObject_Del(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8724 }
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8725 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8726 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8727 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
8728
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8729 void
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8730 PyUnicode_InternInPlace(PyObject **p)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8731 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8732 register PyUnicodeObject *s = (PyUnicodeObject *)(*p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8733 PyObject *t;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8734 if (s == NULL || !PyUnicode_Check(s))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8735 Py_FatalError(
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8736 "PyUnicode_InternInPlace: unicode strings only please!");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8737 /* If it's a subclass, we don't really know what putting
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8738 it in the interned dict might do. */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8739 if (!PyUnicode_CheckExact(s))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8740 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8741 if (PyUnicode_CHECK_INTERNED(s))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8742 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8743 if (interned == NULL) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8744 interned = PyDict_New();
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8745 if (interned == NULL) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8746 PyErr_Clear(); /* Don't leave an exception */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8747 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8748 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8749 }
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8750 /* It might be that the GetItem call fails even
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8751 though the key is present in the dictionary,
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8752 namely when this happens during a stack overflow. */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8753 Py_ALLOW_RECURSION
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8754 t = PyDict_GetItem(interned, (PyObject *)s);
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8755 Py_END_ALLOW_RECURSION
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8756
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8757 if (t) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8758 Py_INCREF(t);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8759 Py_DECREF(*p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8760 *p = t;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8761 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8762 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8763
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8764 PyThreadState_GET()->recursion_critical = 1;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8765 if (PyDict_SetItem(interned, (PyObject *)s, (PyObject *)s) < 0) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8766 PyErr_Clear();
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8767 PyThreadState_GET()->recursion_critical = 0;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8768 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8769 }
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8770 PyThreadState_GET()->recursion_critical = 0;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8771 /* The two references in interned are not counted by refcnt.
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8772 The deallocator will take care of this */
42335
c8b850b5a066 Fix merge breakage.
Martin v. Löwis <martin@v.loewis.de>
parents: 42334
diff changeset
8773 Py_Refcnt(s) -= 2;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8774 PyUnicode_CHECK_INTERNED(s) = SSTATE_INTERNED_MORTAL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8775 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8776
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8777 void
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8778 PyUnicode_InternImmortal(PyObject **p)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8779 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8780 PyUnicode_InternInPlace(p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8781 if (PyUnicode_CHECK_INTERNED(*p) != SSTATE_INTERNED_IMMORTAL) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8782 PyUnicode_CHECK_INTERNED(*p) = SSTATE_INTERNED_IMMORTAL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8783 Py_INCREF(*p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8784 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8785 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8786
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8787 PyObject *
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8788 PyUnicode_InternFromString(const char *cp)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8789 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8790 PyObject *s = PyUnicode_FromString(cp);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8791 if (s == NULL)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8792 return NULL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8793 PyUnicode_InternInPlace(&s);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8794 return s;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8795 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8796
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8797 void _Py_ReleaseInternedUnicodeStrings(void)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8798 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8799 PyObject *keys;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8800 PyUnicodeObject *s;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8801 Py_ssize_t i, n;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8802 Py_ssize_t immortal_size = 0, mortal_size = 0;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8803
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8804 if (interned == NULL || !PyDict_Check(interned))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8805 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8806 keys = PyDict_Keys(interned);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8807 if (keys == NULL || !PyList_Check(keys)) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8808 PyErr_Clear();
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8809 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8810 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8811
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8812 /* Since _Py_ReleaseInternedUnicodeStrings() is intended to help a leak
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8813 detector, interned unicode strings are not forcibly deallocated;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8814 rather, we give them their stolen references back, and then clear
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8815 and DECREF the interned dict. */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8816
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8817 n = PyList_GET_SIZE(keys);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8818 fprintf(stderr, "releasing %" PY_FORMAT_SIZE_T "d interned strings\n",
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8819 n);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8820 for (i = 0; i < n; i++) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8821 s = (PyUnicodeObject *) PyList_GET_ITEM(keys, i);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8822 switch (s->state) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8823 case SSTATE_NOT_INTERNED:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8824 /* XXX Shouldn't happen */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8825 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8826 case SSTATE_INTERNED_IMMORTAL:
42335
c8b850b5a066 Fix merge breakage.
Martin v. Löwis <martin@v.loewis.de>
parents: 42334
diff changeset
8827 Py_Refcnt(s) += 1;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8828 immortal_size += s->length;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8829 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8830 case SSTATE_INTERNED_MORTAL:
42335
c8b850b5a066 Fix merge breakage.
Martin v. Löwis <martin@v.loewis.de>
parents: 42334
diff changeset
8831 Py_Refcnt(s) += 2;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8832 mortal_size += s->length;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8833 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8834 default:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8835 Py_FatalError("Inconsistent interned string state.");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8836 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8837 s->state = SSTATE_NOT_INTERNED;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8838 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8839 fprintf(stderr, "total size of all interned strings: "
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8840 "%" PY_FORMAT_SIZE_T "d/%" PY_FORMAT_SIZE_T "d "
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8841 "mortal/immortal\n", mortal_size, immortal_size);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8842 Py_DECREF(keys);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8843 PyDict_Clear(interned);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8844 Py_DECREF(interned);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8845 interned = NULL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8846 }
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8847
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8848
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8849 /********************* Unicode Iterator **************************/
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8850
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8851 typedef struct {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8852 PyObject_HEAD
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8853 Py_ssize_t it_index;
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8854 PyUnicodeObject *it_seq; /* Set to NULL when iterator is exhausted */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8855 } unicodeiterobject;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8856
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8857 static void
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8858 unicodeiter_dealloc(unicodeiterobject *it)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8859 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8860 _PyObject_GC_UNTRACK(it);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8861 Py_XDECREF(it->it_seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8862 PyObject_GC_Del(it);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8863 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8864
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8865 static int
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8866 unicodeiter_traverse(unicodeiterobject *it, visitproc visit, void *arg)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8867 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8868 Py_VISIT(it->it_seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8869 return 0;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8870 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8871
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8872 static PyObject *
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8873 unicodeiter_next(unicodeiterobject *it)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8874 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8875 PyUnicodeObject *seq;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8876 PyObject *item;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8877
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8878 assert(it != NULL);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8879 seq = it->it_seq;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8880 if (seq == NULL)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8881 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8882 assert(PyUnicode_Check(seq));
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8883
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8884 if (it->it_index < PyUnicode_GET_SIZE(seq)) {
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8885 item = PyUnicode_FromUnicode(
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8886 PyUnicode_AS_UNICODE(seq)+it->it_index, 1);
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8887 if (item != NULL)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8888 ++it->it_index;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8889 return item;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8890 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8891
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8892 Py_DECREF(seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8893 it->it_seq = NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8894 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8895 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8896
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8897 static PyObject *
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8898 unicodeiter_len(unicodeiterobject *it)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8899 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8900 Py_ssize_t len = 0;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8901 if (it->it_seq)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8902 len = PyUnicode_GET_SIZE(it->it_seq) - it->it_index;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8903 return PyInt_FromSsize_t(len);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8904 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8905
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8906 PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of len(list(it)).");
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8907
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8908 static PyMethodDef unicodeiter_methods[] = {
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8909 {"__length_hint__", (PyCFunction)unicodeiter_len, METH_NOARGS,
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8910 length_hint_doc},
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8911 {NULL, NULL} /* sentinel */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8912 };
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8913
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8914 PyTypeObject PyUnicodeIter_Type = {
42334
e70cdce0e8b2 Merged revisions 56467-56482 via svnmerge from
Martin v. Löwis <martin@v.loewis.de>
parents: 42318
diff changeset
8915 PyVarObject_HEAD_INIT(&PyType_Type, 0)
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8916 "unicodeiterator", /* tp_name */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8917 sizeof(unicodeiterobject), /* tp_basicsize */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8918 0, /* tp_itemsize */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8919 /* methods */
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8920 (destructor)unicodeiter_dealloc, /* tp_dealloc */
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8921 0, /* tp_print */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8922 0, /* tp_getattr */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8923 0, /* tp_setattr */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8924 0, /* tp_compare */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8925 0, /* tp_repr */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8926 0, /* tp_as_number */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8927 0, /* tp_as_sequence */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8928 0, /* tp_as_mapping */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8929 0, /* tp_hash */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8930 0, /* tp_call */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8931 0, /* tp_str */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8932 PyObject_GenericGetAttr, /* tp_getattro */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8933 0, /* tp_setattro */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8934 0, /* tp_as_buffer */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8935 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8936 0, /* tp_doc */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8937 (traverseproc)unicodeiter_traverse, /* tp_traverse */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8938 0, /* tp_clear */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8939 0, /* tp_richcompare */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8940 0, /* tp_weaklistoffset */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8941 PyObject_SelfIter, /* tp_iter */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8942 (iternextfunc)unicodeiter_next, /* tp_iternext */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8943 unicodeiter_methods, /* tp_methods */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8944 0,
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8945 };
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8946
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8947 static PyObject *
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8948 unicode_iter(PyObject *seq)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8949 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8950 unicodeiterobject *it;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8951
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8952 if (!PyUnicode_Check(seq)) {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8953 PyErr_BadInternalCall();
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8954 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8955 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8956 it = PyObject_GC_New(unicodeiterobject, &PyUnicodeIter_Type);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8957 if (it == NULL)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8958 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8959 it->it_index = 0;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8960 Py_INCREF(seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8961 it->it_seq = (PyUnicodeObject *)seq;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8962 _PyObject_GC_TRACK(it);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8963 return (PyObject *)it;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8964 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8965
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8966 size_t
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8967 Py_UNICODE_strlen(const Py_UNICODE *u)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8968 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8969 int res = 0;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8970 while(*u++)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8971 res++;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8972 return res;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8973 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8974
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8975 Py_UNICODE*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8976 Py_UNICODE_strcpy(Py_UNICODE *s1, const Py_UNICODE *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8977 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8978 Py_UNICODE *u = s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8979 while ((*u++ = *s2++));
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8980 return s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8981 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8982
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8983 Py_UNICODE*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8984 Py_UNICODE_strncpy(Py_UNICODE *s1, const Py_UNICODE *s2, size_t n)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8985 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8986 Py_UNICODE *u = s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8987 while ((*u++ = *s2++))
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8988 if (n-- == 0)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8989 break;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8990 return s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8991 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8992
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8993 int
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8994 Py_UNICODE_strcmp(const Py_UNICODE *s1, const Py_UNICODE *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8995 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8996 while (*s1 && *s2 && *s1 == *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8997 s1++, s2++;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8998 if (*s1 && *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8999 return (*s1 < *s2) ? -1 : +1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9000 if (*s1)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9001 return 1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9002 if (*s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9003 return -1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9004 return 0;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9005 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9006
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9007 Py_UNICODE*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9008 Py_UNICODE_strchr(const Py_UNICODE *s, Py_UNICODE c)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9009 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9010 const Py_UNICODE *p;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9011 for (p = s; *p; p++)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9012 if (*p == c)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9013 return (Py_UNICODE*)p;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9014 return NULL;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9015 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9016
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9017
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9018 #ifdef __cplusplus
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9019 }
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9020 #endif
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9021
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9022
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9023 /*
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9024 Local variables:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9025 c-basic-offset: 4
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9026 indent-tabs-mode: nil
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9027 End:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9028 */