Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodeobject.c @ 42257:c280647c330c

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
c_char, c_char_p objects and c_char array structure fields return their value now as str, no longer str8.
author Thomas Heller <theller@ctypes.org>
date Fri, 13 Jul 2007 12:52:51 +0000
parents ca3cfb34e3c2
children 18e5ae91bd29
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1 /*
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 Unicode implementation based on original code by Fredrik Lundh,
12646
17d1a3f1f085 Minimal change so I can add the rest of MAL's checkin message:
Fred Drake <fdrake@acm.org>
parents: 12645
diff changeset
4 modified by Marc-Andre Lemburg <mal@lemburg.com> according to the
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 Unicode Integration Proposal (see file Misc/unicode.txt).
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7 Major speed upgrades to the method implementations at the Reykjavik
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8 NeedForSpeed sprint, by Fredrik Lundh and Andrew Dalke.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
9
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13761
diff changeset
10 Copyright (c) Corporation for National Research Initiatives.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
11
16029
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
12 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
13 The original string type implementation is:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
14
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
15 Copyright (c) 1999 by Secret Labs AB
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
16 Copyright (c) 1999 by Fredrik Lundh
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
17
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
18 By obtaining, using, and/or copying this software and/or its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
19 associated documentation, you agree that you have read, understood,
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
20 and will comply with the following terms and conditions:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
21
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
22 Permission to use, copy, modify, and distribute this software and its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
23 associated documentation for any purpose and without fee is hereby
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
24 granted, provided that the above copyright notice appears in all
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
25 copies, and that both that copyright notice and this permission notice
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
26 appear in supporting documentation, and that the name of Secret Labs
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
27 AB or the author not be used in advertising or publicity pertaining to
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
28 distribution of the software without specific, written prior
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
29 permission.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
30
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
31 SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
32 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
33 FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
34 ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
35 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
36 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
37 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
38 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
39
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
40 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
41
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
42 #define PY_SSIZE_T_CLEAN
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
43 #include "Python.h"
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
44
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
45 #include "unicodeobject.h"
13209
b11e0a898de0 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13196
diff changeset
46 #include "ucnhash.h"
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
47
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
48 #ifdef MS_WINDOWS
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
49 #include <windows.h>
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
50 #endif
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
51
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 /* Limit for the Unicode object free list */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
53
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 #define MAX_UNICODE_FREELIST_SIZE 1024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
55
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
56 /* Limit for the Unicode object free list stay alive optimization.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
57
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
58 The implementation will keep allocated Unicode memory intact for
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
59 all objects on the free list having a size less than this
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
60 limit. This reduces malloc() overhead for small Unicode objects.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
61
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
62 At worst this will result in MAX_UNICODE_FREELIST_SIZE *
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
63 (sizeof(PyUnicodeObject) + KEEPALIVE_SIZE_LIMIT +
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
64 malloc()-overhead) bytes of unused garbage.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
65
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
66 Setting the limit to 0 effectively turns the feature off.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
67
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
68 Note: This is an experimental feature ! If you get core dumps when
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
69 using Unicode objects, turn this feature off.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
70
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
72
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
73 #define KEEPALIVE_SIZE_LIMIT 9
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
74
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 /* Endianness switches; defaults to little endian */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
76
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 #ifdef WORDS_BIGENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
78 # define BYTEORDER_IS_BIG_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
79 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
80 # define BYTEORDER_IS_LITTLE_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
81 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
82
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
83 /* --- Globals ------------------------------------------------------------
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
84
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
85 The globals are initialized by the _PyUnicode_Init() API and should
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
86 not be used before calling that API.
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
87
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
88 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
89
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
90
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
91 #ifdef __cplusplus
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
92 extern "C" {
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
93 #endif
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
94
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
95 /* This dictionary holds all interned unicode strings. Note that references
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
96 to strings in this dictionary are *not* counted in the string's ob_refcnt.
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
97 When the interned string reaches a refcnt of 0 the string deallocation
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
98 function will delete the reference from this dictionary.
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
99
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
100 Another way to look at this is that to say that the actual reference
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
101 count of a string is: s->ob_refcnt + (s->ob_sstate?2:0)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
102 */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
103 static PyObject *interned;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
104
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
105 /* Free list for Unicode objects */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
106 static PyUnicodeObject *unicode_freelist;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
107 static int unicode_freelist_size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
108
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
109 /* The empty Unicode object is shared to improve performance. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
110 static PyUnicodeObject *unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
111
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
112 /* Single character Unicode strings in the Latin-1 range are being
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
113 shared as well. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
114 static PyUnicodeObject *unicode_latin1[256];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
115
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
116 /* Default encoding to use and assume when NULL is passed as encoding
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
117 parameter; it is fixed to "utf-8". Always use the
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
118 PyUnicode_GetDefaultEncoding() API to access this global. */
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
119 static const char unicode_default_encoding[] = "utf-8";
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
120
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
121 Py_UNICODE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
122 PyUnicode_GetMax(void)
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
123 {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
124 #ifdef Py_UNICODE_WIDE
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
125 return 0x10FFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
126 #else
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
127 /* This is actually an illegal character, so it should
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
128 not be passed to unichr. */
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
129 return 0xFFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
130 #endif
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
131 }
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
132
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
133 /* --- Bloom Filters ----------------------------------------------------- */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
134
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
135 /* stuff to implement simple "bloom filters" for Unicode characters.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
136 to keep things simple, we use a single bitmask, using the least 5
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
137 bits from each unicode characters as the bit index. */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
138
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
139 /* the linebreak mask is set up by Unicode_Init below */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
140
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
141 #define BLOOM_MASK unsigned long
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
142
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
143 static BLOOM_MASK bloom_linebreak;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
144
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
145 #define BLOOM(mask, ch) ((mask & (1 << ((ch) & 0x1F))))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
146
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
147 #define BLOOM_LINEBREAK(ch)\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
148 (BLOOM(bloom_linebreak, (ch)) && Py_UNICODE_ISLINEBREAK((ch)))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
149
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
150 Py_LOCAL_INLINE(BLOOM_MASK) make_bloom_mask(Py_UNICODE* ptr, Py_ssize_t len)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
151 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
152 /* calculate simple bloom-style bitmask for a given unicode string */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
153
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
154 long mask;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
155 Py_ssize_t i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
156
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
157 mask = 0;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
158 for (i = 0; i < len; i++)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
159 mask |= (1 << (ptr[i] & 0x1F));
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
160
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
161 return mask;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
162 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
163
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
164 Py_LOCAL_INLINE(int) unicode_member(Py_UNICODE chr, Py_UNICODE* set, Py_ssize_t setlen)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
165 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
166 Py_ssize_t i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
167
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
168 for (i = 0; i < setlen; i++)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
169 if (set[i] == chr)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
170 return 1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
171
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
172 return 0;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
173 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
174
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
175 #define BLOOM_MEMBER(mask, chr, set, setlen)\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
176 BLOOM(mask, chr) && unicode_member(chr, set, setlen)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
177
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
178 /* --- Unicode Object ----------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
179
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
180 static
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
181 int unicode_resize(register PyUnicodeObject *unicode,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
182 Py_ssize_t length)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
183 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
184 void *oldstr;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
185
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
186 /* Shortcut if there's nothing much to do. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
187 if (unicode->length == length)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
188 goto reset;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
189
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
190 /* Resizing shared object (unicode_empty or single character
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
191 objects) in-place is not allowed. Use PyUnicode_Resize()
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
192 instead ! */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
193
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
194 if (unicode == unicode_empty ||
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
195 (unicode->length == 1 &&
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
196 unicode->str[0] < 256U &&
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
197 unicode_latin1[unicode->str[0]] == unicode)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
198 PyErr_SetString(PyExc_SystemError,
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
199 "can't resize shared unicode objects");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
200 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
202
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
203 /* We allocate one more byte to make sure the string is Ux0000 terminated.
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
204 The overallocation is also used by fastsearch, which assumes that it's
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
205 safe to look at str[length] (without making any assumptions about what
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
206 it contains). */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
207
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
208 oldstr = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
209 PyMem_RESIZE(unicode->str, Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
210 if (!unicode->str) {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
211 unicode->str = (Py_UNICODE *)oldstr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
212 PyErr_NoMemory();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
213 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
214 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
215 unicode->str[length] = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
216 unicode->length = length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
217
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
218 reset:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
219 /* Reset the object caches */
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
220 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
221 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
222 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
223 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
224 unicode->hash = -1;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
225
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
226 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
227 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
228
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
229 /* We allocate one more byte to make sure the string is
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
230 Ux0000 terminated -- XXX is this needed ?
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
231
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
232 XXX This allocator could further be enhanced by assuring that the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
233 free list never reduces its size below 1.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
234
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
235 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
236
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
237 static
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
238 PyUnicodeObject *_PyUnicode_New(Py_ssize_t length)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
239 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
240 register PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
241
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
242 /* Optimization for empty strings */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
243 if (length == 0 && unicode_empty != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
244 Py_INCREF(unicode_empty);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
245 return unicode_empty;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
246 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
247
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
248 /* Unicode freelist & memory allocation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
249 if (unicode_freelist) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
250 unicode = unicode_freelist;
12939
88004f25dc25 Vladimir MARANGOZOV <Vladimir.Marangozov@inrialpes.fr>:
Marc-André Lemburg <mal@egenix.com>
parents: 12924
diff changeset
251 unicode_freelist = *(PyUnicodeObject **)unicode;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
252 unicode_freelist_size--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
253 if (unicode->str) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
254 /* Keep-Alive optimization: we only upsize the buffer,
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
255 never downsize it. */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
256 if ((unicode->length < length) &&
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
257 unicode_resize(unicode, length) < 0) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
258 PyMem_DEL(unicode->str);
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
259 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
260 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
261 }
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
262 else {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
263 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
264 }
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
265 PyObject_INIT(unicode, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
266 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
267 else {
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
268 unicode = PyObject_New(PyUnicodeObject, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
269 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
270 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
271 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
272 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
273
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
274 if (!unicode->str) {
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
275 PyErr_NoMemory();
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
276 goto onError;
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
277 }
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
278 /* Initialize the first element to guard against cases where
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
279 * the caller fails before initializing str -- unicode_resize()
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
280 * reads str[0], and the Keep-Alive optimization can keep memory
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
281 * allocated for str alive across a call to unicode_dealloc(unicode).
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
282 * We don't want unicode_resize to read uninitialized memory in
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
283 * that case.
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
284 */
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
285 unicode->str[0] = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
286 unicode->str[length] = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
287 unicode->length = length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
288 unicode->hash = -1;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
289 unicode->state = 0;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
290 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
291 return unicode;
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
292
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
293 onError:
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
294 _Py_ForgetReference((PyObject *)unicode);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
295 PyObject_Del(unicode);
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
296 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
297 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
298
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
299 static
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
300 void unicode_dealloc(register PyUnicodeObject *unicode)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
301 {
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
302 switch (PyUnicode_CHECK_INTERNED(unicode)) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
303 case SSTATE_NOT_INTERNED:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
304 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
305
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
306 case SSTATE_INTERNED_MORTAL:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
307 /* revive dead object temporarily for DelItem */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
308 unicode->ob_refcnt = 3;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
309 if (PyDict_DelItem(interned, (PyObject *)unicode) != 0)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
310 Py_FatalError(
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
311 "deletion of interned unicode string failed");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
312 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
313
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
314 case SSTATE_INTERNED_IMMORTAL:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
315 Py_FatalError("Immortal interned unicode string died.");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
316
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
317 default:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
318 Py_FatalError("Inconsistent interned unicode string state.");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
319 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
320
21262
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
321 if (PyUnicode_CheckExact(unicode) &&
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
322 unicode_freelist_size < MAX_UNICODE_FREELIST_SIZE) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
323 /* Keep-Alive optimization */
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
324 if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
325 PyMem_DEL(unicode->str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
326 unicode->str = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
327 unicode->length = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
328 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
329 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
330 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
331 unicode->defenc = NULL;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
332 }
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
333 /* Add to free list */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
334 *(PyUnicodeObject **)unicode = unicode_freelist;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
335 unicode_freelist = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
336 unicode_freelist_size++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
337 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
338 else {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
339 PyMem_DEL(unicode->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
340 Py_XDECREF(unicode->defenc);
21262
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
341 unicode->ob_type->tp_free((PyObject *)unicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
342 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
343 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
344
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
345 int PyUnicode_Resize(PyObject **unicode, Py_ssize_t length)
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
346 {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
347 register PyUnicodeObject *v;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
348
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
349 /* Argument checks */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
350 if (unicode == NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
351 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
352 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
353 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
354 v = (PyUnicodeObject *)*unicode;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
355 if (v == NULL || !PyUnicode_Check(v) || v->ob_refcnt != 1 || length < 0) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
356 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
357 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
358 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
359
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
360 /* Resizing unicode_empty and single character objects is not
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
361 possible since these are being shared. We simply return a fresh
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
362 copy with the same Unicode content. */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
363 if (v->length != length &&
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
364 (v == unicode_empty || v->length == 1)) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
365 PyUnicodeObject *w = _PyUnicode_New(length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
366 if (w == NULL)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
367 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
368 Py_UNICODE_COPY(w->str, v->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
369 length < v->length ? length : v->length);
28242
fe1d571c3f80 Sf patch #700047: unicode object leaks refcount on resizing
Raymond Hettinger <python@rcn.com>
parents: 27826
diff changeset
370 Py_DECREF(*unicode);
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
371 *unicode = (PyObject *)w;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
372 return 0;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
373 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
374
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
375 /* Note that we don't have to modify *unicode for unshared Unicode
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
376 objects, since we can modify them in-place. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
377 return unicode_resize(v, length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
378 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
379
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
380 /* Internal API for use in unicodeobject.c only ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
381 #define _PyUnicode_Resize(unicodevar, length) \
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
382 PyUnicode_Resize(((PyObject **)(unicodevar)), length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
383
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
384 PyObject *PyUnicode_FromUnicode(const Py_UNICODE *u,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
385 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
386 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
387 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
388
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
389 /* If the Unicode data is known at construction time, we can apply
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
390 some optimizations which share commonly used objects. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
391 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
392
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
393 /* Optimization for empty strings */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
394 if (size == 0 && unicode_empty != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
395 Py_INCREF(unicode_empty);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
396 return (PyObject *)unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
397 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
398
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
399 /* Single character Unicode objects in the Latin-1 range are
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
400 shared when using this constructor */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
401 if (size == 1 && *u < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
402 unicode = unicode_latin1[*u];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
403 if (!unicode) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
404 unicode = _PyUnicode_New(1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
405 if (!unicode)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
406 return NULL;
18051
e897550b4748 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode.
Marc-André Lemburg <mal@egenix.com>
parents: 17995
diff changeset
407 unicode->str[0] = *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
408 unicode_latin1[*u] = unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
409 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
410 Py_INCREF(unicode);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
411 return (PyObject *)unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
412 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
413 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
414
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
415 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
416 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
417 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
418
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
419 /* Copy the Unicode data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
420 if (u != NULL)
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
421 Py_UNICODE_COPY(unicode->str, u, size);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
422
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
423 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
424 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
425
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
426 PyObject *PyUnicode_FromStringAndSize(const char *u, Py_ssize_t size)
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
427 {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
428 PyUnicodeObject *unicode;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
429 /* If the Unicode data is known at construction time, we can apply
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
430 some optimizations which share commonly used objects. */
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
431 if (u != NULL) {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
432
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
433 /* Optimization for empty strings */
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
434 if (size == 0 && unicode_empty != NULL) {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
435 Py_INCREF(unicode_empty);
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
436 return (PyObject *)unicode_empty;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
437 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
438
41622
107aacb9c595 When creating a unicode object from a char * characters
Walter Dörwald <walter@livinglogic.de>
parents: 41619
diff changeset
439 /* Single characters are shared when using this constructor */
107aacb9c595 When creating a unicode object from a char * characters
Walter Dörwald <walter@livinglogic.de>
parents: 41619
diff changeset
440 if (size == 1) {
41623
6bc156d3c00e Silence gcc warnings.
Walter Dörwald <walter@livinglogic.de>
parents: 41622
diff changeset
441 unicode = unicode_latin1[(int)*u];
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
442 if (!unicode) {
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
443 unicode = _PyUnicode_New(1);
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
444 if (!unicode)
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
445 return NULL;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
446 unicode->str[0] = *u;
41623
6bc156d3c00e Silence gcc warnings.
Walter Dörwald <walter@livinglogic.de>
parents: 41622
diff changeset
447 unicode_latin1[(int)*u] = unicode;
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
448 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
449 Py_INCREF(unicode);
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
450 return (PyObject *)unicode;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
451 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
452 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
453
41765
141e2aeffbaa Revert last checkin: _PyUnicode_New() allocates space
Walter Dörwald <walter@livinglogic.de>
parents: 41764
diff changeset
454 unicode = _PyUnicode_New(size);
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
455 if (!unicode)
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
456 return NULL;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
457
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
458 /* Copy the Unicode data into the new object */
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
459 if (u != NULL) {
41623
6bc156d3c00e Silence gcc warnings.
Walter Dörwald <walter@livinglogic.de>
parents: 41622
diff changeset
460 Py_UNICODE *p = unicode->str;
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
461 while (size--)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
462 *p++ = *u++;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
463 /* Don't need to write trailing 0 because
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
464 that's already done by _PyUnicode_New */
41619
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
465 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
466
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
467 return (PyObject *)unicode;
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
468 }
61acfb831633 Add PyUnicode_FromString(), which create a unicode object from a
Walter Dörwald <walter@livinglogic.de>
parents: 41605
diff changeset
469
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
470 PyObject *PyUnicode_FromString(const char *u)
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
471 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
472 size_t size = strlen(u);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
473 if (size > PY_SSIZE_T_MAX) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
474 PyErr_SetString(PyExc_OverflowError, "input too long");
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
475 return NULL;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
476 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
477
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
478 return PyUnicode_FromStringAndSize(u, size);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
479 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
480
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
481 #ifdef HAVE_WCHAR_H
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
482
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
483 PyObject *PyUnicode_FromWideChar(register const wchar_t *w,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
484 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
485 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
486 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
487
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
488 if (w == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
489 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
490 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
491 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
492
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
493 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
494 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
495 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
496
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
497 /* Copy the wchar_t data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
498 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
499 memcpy(unicode->str, w, size * sizeof(wchar_t));
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
500 #else
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
501 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
502 register Py_UNICODE *u;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
503 register Py_ssize_t i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
504 u = PyUnicode_AS_UNICODE(unicode);
33766
1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
Marc-André Lemburg <mal@egenix.com>
parents: 33499
diff changeset
505 for (i = size; i > 0; i--)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
506 *u++ = *w++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
507 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
508 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
509
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
510 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
511 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
512
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
513 static void
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
514 makefmt(char *fmt, int longflag, int size_tflag, int zeropad, int width, int precision, char c)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
515 {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
516 *fmt++ = '%';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
517 if (width) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
518 if (zeropad)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
519 *fmt++ = '0';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
520 fmt += sprintf(fmt, "%d", width);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
521 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
522 if (precision)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
523 fmt += sprintf(fmt, ".%d", precision);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
524 if (longflag)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
525 *fmt++ = 'l';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
526 else if (size_tflag) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
527 char *f = PY_FORMAT_SIZE_T;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
528 while (*f)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
529 *fmt++ = *f++;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
530 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
531 *fmt++ = c;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
532 *fmt = '\0';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
533 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
534
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
535 #define appendstring(string) {for (copy = string;*copy;) *s++ = *copy++;}
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
536
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
537 PyObject *
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
538 PyUnicode_FromFormatV(const char *format, va_list vargs)
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
539 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
540 va_list count;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
541 Py_ssize_t callcount = 0;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
542 PyObject **callresults = NULL;
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
543 PyObject **callresult = NULL;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
544 Py_ssize_t n = 0;
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
545 int width = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
546 int precision = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
547 int zeropad;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
548 const char* f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
549 Py_UNICODE *s;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
550 PyObject *string;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
551 /* used by sprintf */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
552 char buffer[21];
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
553 /* use abuffer instead of buffer, if we need more space
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
554 * (which can happen if there's a format specifier with width). */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
555 char *abuffer = NULL;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
556 char *realbuffer;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
557 Py_ssize_t abuffersize = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
558 char fmt[60]; /* should be enough for %0width.precisionld */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
559 const char *copy;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
560
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
561 #ifdef VA_LIST_IS_ARRAY
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
562 Py_MEMCPY(count, vargs, sizeof(va_list));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
563 #else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
564 #ifdef __va_copy
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
565 __va_copy(count, vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
566 #else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
567 count = vargs;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
568 #endif
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
569 #endif
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
570 /* step 1: count the number of %S/%R format specifications
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
571 * (we call PyObject_Unicode()/PyObject_Repr() for these objects
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
572 * once during step 3 and put the result in an array) */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
573 for (f = format; *f; f++) {
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
574 if (*f == '%' && (*(f+1)=='S' || *(f+1)=='R'))
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
575 ++callcount;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
576 }
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
577 /* step 2: allocate memory for the results of
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
578 * PyObject_Unicode()/PyObject_Repr() calls */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
579 if (callcount) {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
580 callresults = PyMem_Malloc(sizeof(PyObject *)*callcount);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
581 if (!callresults) {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
582 PyErr_NoMemory();
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
583 return NULL;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
584 }
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
585 callresult = callresults;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
586 }
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
587 /* step 3: figure out how large a buffer we need */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
588 for (f = format; *f; f++) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
589 if (*f == '%') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
590 const char* p = f;
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
591 width = 0;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
592 while (isdigit(Py_CHARMASK(*f)))
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
593 width = (width*10) + *f++ - '0';
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
594 while (*++f && *f != '%' && !isalpha(Py_CHARMASK(*f)))
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
595 ;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
596
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
597 /* skip the 'l' or 'z' in {%ld, %zd, %lu, %zu} since
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
598 * they don't affect the amount of space we reserve.
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
599 */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
600 if ((*f == 'l' || *f == 'z') &&
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
601 (f[1] == 'd' || f[1] == 'u'))
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
602 ++f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
603
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
604 switch (*f) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
605 case 'c':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
606 (void)va_arg(count, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
607 /* fall through... */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
608 case '%':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
609 n++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
610 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
611 case 'd': case 'u': case 'i': case 'x':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
612 (void) va_arg(count, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
613 /* 20 bytes is enough to hold a 64-bit
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
614 integer. Decimal takes the most space.
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
615 This isn't enough for octal.
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
616 If a width is specified we need more
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
617 (which we allocate later). */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
618 if (width < 20)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
619 width = 20;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
620 n += width;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
621 if (abuffersize < width)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
622 abuffersize = width;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
623 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
624 case 's':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
625 n += strlen(va_arg(count, char*));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
626 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
627 case 'U':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
628 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
629 PyObject *obj = va_arg(count, PyObject *);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
630 assert(obj && PyUnicode_Check(obj));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
631 n += PyUnicode_GET_SIZE(obj);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
632 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
633 }
42047
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
634 case 'V':
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
635 {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
636 PyObject *obj = va_arg(count, PyObject *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
637 const char *str = va_arg(count, const char *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
638 assert(obj || str);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
639 assert(!obj || PyUnicode_Check(obj));
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
640 if (obj)
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
641 n += PyUnicode_GET_SIZE(obj);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
642 else
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
643 n += strlen(str);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
644 break;
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
645 }
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
646 case 'S':
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
647 {
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
648 PyObject *obj = va_arg(count, PyObject *);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
649 PyObject *str;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
650 assert(obj);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
651 str = PyObject_Unicode(obj);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
652 if (!str)
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
653 goto fail;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
654 n += PyUnicode_GET_SIZE(str);
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
655 /* Remember the str and switch to the next slot */
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
656 *callresult++ = str;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
657 break;
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
658 }
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
659 case 'R':
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
660 {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
661 PyObject *obj = va_arg(count, PyObject *);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
662 PyObject *repr;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
663 assert(obj);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
664 repr = PyObject_Repr(obj);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
665 if (!repr)
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
666 goto fail;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
667 n += PyUnicode_GET_SIZE(repr);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
668 /* Remember the repr and switch to the next slot */
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
669 *callresult++ = repr;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
670 break;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
671 }
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
672 case 'p':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
673 (void) va_arg(count, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
674 /* maximum 64-bit pointer representation:
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
675 * 0xffffffffffffffff
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
676 * so 19 characters is enough.
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
677 * XXX I count 18 -- what's the extra for?
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
678 */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
679 n += 19;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
680 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
681 default:
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
682 /* if we stumble upon an unknown
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
683 formatting code, copy the rest of
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
684 the format string to the output
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
685 string. (we cannot just skip the
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
686 code, since there's no way to know
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
687 what's in the argument list) */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
688 n += strlen(p);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
689 goto expand;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
690 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
691 } else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
692 n++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
693 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
694 expand:
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
695 if (abuffersize > 20) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
696 abuffer = PyMem_Malloc(abuffersize);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
697 if (!abuffer) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
698 PyErr_NoMemory();
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
699 goto fail;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
700 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
701 realbuffer = abuffer;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
702 }
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
703 else
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
704 realbuffer = buffer;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
705 /* step 4: fill the buffer */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
706 /* Since we've analyzed how much space we need for the worst case,
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
707 we don't have to resize the string.
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
708 There can be no errors beyond this point. */
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
709 string = PyUnicode_FromUnicode(NULL, n);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
710 if (!string)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
711 goto fail;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
712
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
713 s = PyUnicode_AS_UNICODE(string);
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
714 callresult = callresults;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
715
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
716 for (f = format; *f; f++) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
717 if (*f == '%') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
718 const char* p = f++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
719 int longflag = 0;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
720 int size_tflag = 0;
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
721 zeropad = (*f == '0');
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
722 /* parse the width.precision part */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
723 width = 0;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
724 while (isdigit(Py_CHARMASK(*f)))
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
725 width = (width*10) + *f++ - '0';
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
726 precision = 0;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
727 if (*f == '.') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
728 f++;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
729 while (isdigit(Py_CHARMASK(*f)))
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
730 precision = (precision*10) + *f++ - '0';
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
731 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
732 /* handle the long flag, but only for %ld and %lu.
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
733 others can be added when necessary. */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
734 if (*f == 'l' && (f[1] == 'd' || f[1] == 'u')) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
735 longflag = 1;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
736 ++f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
737 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
738 /* handle the size_t flag. */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
739 if (*f == 'z' && (f[1] == 'd' || f[1] == 'u')) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
740 size_tflag = 1;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
741 ++f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
742 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
743
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
744 switch (*f) {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
745 case 'c':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
746 *s++ = va_arg(vargs, int);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
747 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
748 case 'd':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
749 makefmt(fmt, longflag, size_tflag, zeropad, width, precision, 'd');
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
750 if (longflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
751 sprintf(realbuffer, fmt, va_arg(vargs, long));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
752 else if (size_tflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
753 sprintf(realbuffer, fmt, va_arg(vargs, Py_ssize_t));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
754 else
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
755 sprintf(realbuffer, fmt, va_arg(vargs, int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
756 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
757 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
758 case 'u':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
759 makefmt(fmt, longflag, size_tflag, zeropad, width, precision, 'u');
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
760 if (longflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
761 sprintf(realbuffer, fmt, va_arg(vargs, unsigned long));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
762 else if (size_tflag)
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
763 sprintf(realbuffer, fmt, va_arg(vargs, size_t));
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
764 else
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
765 sprintf(realbuffer, fmt, va_arg(vargs, unsigned int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
766 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
767 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
768 case 'i':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
769 makefmt(fmt, 0, 0, zeropad, width, precision, 'i');
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
770 sprintf(realbuffer, fmt, va_arg(vargs, int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
771 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
772 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
773 case 'x':
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
774 makefmt(fmt, 0, 0, zeropad, width, precision, 'x');
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
775 sprintf(realbuffer, fmt, va_arg(vargs, int));
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
776 appendstring(realbuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
777 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
778 case 's':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
779 p = va_arg(vargs, char*);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
780 appendstring(p);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
781 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
782 case 'U':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
783 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
784 PyObject *obj = va_arg(vargs, PyObject *);
41861
969868df1938 Simplify %U handling by using Py_UNICODE_COPY.
Walter Dörwald <walter@livinglogic.de>
parents: 41840
diff changeset
785 Py_ssize_t size = PyUnicode_GET_SIZE(obj);
969868df1938 Simplify %U handling by using Py_UNICODE_COPY.
Walter Dörwald <walter@livinglogic.de>
parents: 41840
diff changeset
786 Py_UNICODE_COPY(s, PyUnicode_AS_UNICODE(obj), size);
969868df1938 Simplify %U handling by using Py_UNICODE_COPY.
Walter Dörwald <walter@livinglogic.de>
parents: 41840
diff changeset
787 s += size;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
788 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
789 }
42047
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
790 case 'V':
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
791 {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
792 PyObject *obj = va_arg(vargs, PyObject *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
793 const char *str = va_arg(vargs, const char *);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
794 if (obj) {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
795 Py_ssize_t size = PyUnicode_GET_SIZE(obj);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
796 Py_UNICODE_COPY(s, PyUnicode_AS_UNICODE(obj), size);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
797 s += size;
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
798 } else {
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
799 appendstring(str);
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
800 }
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
801 break;
d678ba0177da Add a format specifier %V to PyUnicode_FromFormat(), that works similar to %U,
Walter Dörwald <walter@livinglogic.de>
parents: 42013
diff changeset
802 }
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
803 case 'S':
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
804 case 'R':
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
805 {
42073
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
806 Py_UNICODE *ucopy;
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
807 Py_ssize_t usize;
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
808 Py_ssize_t upos;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
809 /* unused, since we already have the result */
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
810 (void) va_arg(vargs, PyObject *);
42073
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
811 ucopy = PyUnicode_AS_UNICODE(*callresult);
f833b43c2756 Make it compile with GCC 2.96.
Guido van Rossum <guido@python.org>
parents: 42065
diff changeset
812 usize = PyUnicode_GET_SIZE(*callresult);
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
813 for (upos = 0; upos<usize;)
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
814 *s++ = ucopy[upos++];
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
815 /* We're done with the unicode()/repr() => forget it */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
816 Py_DECREF(*callresult);
41840
bfe953046d05 Add a format character %S to PyUnicode_FromFormat() that
Walter Dörwald <walter@livinglogic.de>
parents: 41782
diff changeset
817 /* switch to next unicode()/repr() result */
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
818 ++callresult;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
819 break;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
820 }
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
821 case 'p':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
822 sprintf(buffer, "%p", va_arg(vargs, void*));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
823 /* %p is ill-defined: ensure leading 0x. */
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
824 if (buffer[1] == 'X')
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
825 buffer[1] = 'x';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
826 else if (buffer[1] != 'x') {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
827 memmove(buffer+2, buffer, strlen(buffer)+1);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
828 buffer[0] = '0';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
829 buffer[1] = 'x';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
830 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
831 appendstring(buffer);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
832 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
833 case '%':
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
834 *s++ = '%';
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
835 break;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
836 default:
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
837 appendstring(p);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
838 goto end;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
839 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
840 } else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
841 *s++ = *f;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
842 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
843
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
844 end:
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
845 if (callresults)
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
846 PyMem_Free(callresults);
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
847 if (abuffer)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
848 PyMem_Free(abuffer);
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
849 _PyUnicode_Resize(&string, s - PyUnicode_AS_UNICODE(string));
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
850 return string;
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
851 fail:
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
852 if (callresults) {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
853 PyObject **callresult2 = callresults;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
854 while (callresult2 <= callresult) {
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
855 Py_DECREF(*callresult2);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
856 ++callresult2;
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
857 }
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
858 PyMem_Free(callresults);
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
859 }
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
860 if (abuffer)
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
861 PyMem_Free(abuffer);
41782
bee8d01ef42c Add a format specifier %R to PyUnicode_FromFormat(), which embeds
Walter Dörwald <walter@livinglogic.de>
parents: 41769
diff changeset
862 return NULL;
41766
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
863 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
864
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
865 #undef appendstring
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
866
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
867 PyObject *
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
868 PyUnicode_FromFormat(const char *format, ...)
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
869 {
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
870 PyObject* ret;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
871 va_list vargs;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
872
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
873 #ifdef HAVE_STDARG_PROTOTYPES
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
874 va_start(vargs, format);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
875 #else
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
876 va_start(vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
877 #endif
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
878 ret = PyUnicode_FromFormatV(format, vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
879 va_end(vargs);
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
880 return ret;
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
881 }
d6aa7edf217a Add 'U'/'U#' format characters to Py_BuildValue (and thus
Walter Dörwald <walter@livinglogic.de>
parents: 41765
diff changeset
882
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
883 Py_ssize_t PyUnicode_AsWideChar(PyUnicodeObject *unicode,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
884 wchar_t *w,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
885 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
886 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
887 if (unicode == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
888 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
889 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
890 }
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
891
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
892 /* If possible, try to copy the 0-termination as well */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
893 if (size > PyUnicode_GET_SIZE(unicode))
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
894 size = PyUnicode_GET_SIZE(unicode) + 1;
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
895
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
896 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
897 memcpy(w, unicode->str, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
898 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
899 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
900 register Py_UNICODE *u;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
901 register Py_ssize_t i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
902 u = PyUnicode_AS_UNICODE(unicode);
33766
1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
Marc-André Lemburg <mal@egenix.com>
parents: 33499
diff changeset
903 for (i = size; i > 0; i--)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
904 *w++ = *u++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
905 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
906 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
907
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
908 if (size > PyUnicode_GET_SIZE(unicode))
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
909 return PyUnicode_GET_SIZE(unicode);
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
910 else
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
911 return size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
912 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
913
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
914 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
915
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
916 PyObject *PyUnicode_FromOrdinal(int ordinal)
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
917 {
31873
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
918 Py_UNICODE s[1];
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
919
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
920 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
921 if (ordinal < 0 || ordinal > 0x10ffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
922 PyErr_SetString(PyExc_ValueError,
41954
ef37431ebf2c unichr() is named chr() now => fix name in error message.
Walter Dörwald <walter@livinglogic.de>
parents: 41919
diff changeset
923 "chr() arg not in range(0x110000) "
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
924 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
925 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
926 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
927 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
928 if (ordinal < 0 || ordinal > 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
929 PyErr_SetString(PyExc_ValueError,
41954
ef37431ebf2c unichr() is named chr() now => fix name in error message.
Walter Dörwald <walter@livinglogic.de>
parents: 41919
diff changeset
930 "chr() arg not in range(0x10000) "
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
931 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
932 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
933 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
934 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
935
31873
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
936 s[0] = (Py_UNICODE)ordinal;
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
937 return PyUnicode_FromUnicode(s, 1);
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
938 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
939
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
940 PyObject *PyUnicode_FromObject(register PyObject *obj)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
941 {
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
942 /* XXX Perhaps we should make this API an alias of
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
943 PyObject_Unicode() instead ?! */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
944 if (PyUnicode_CheckExact(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
945 Py_INCREF(obj);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
946 return obj;
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
947 }
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
948 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
949 /* For a Unicode subtype that's not a Unicode object,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
950 return a true Unicode object with the same data. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
951 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(obj),
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
952 PyUnicode_GET_SIZE(obj));
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
953 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
954 return PyUnicode_FromEncodedObject(obj, NULL, "strict");
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
955 }
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
956
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
957 PyObject *PyUnicode_FromEncodedObject(register PyObject *obj,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
958 const char *encoding,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
959 const char *errors)
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
960 {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
961 const char *s = NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
962 Py_ssize_t len;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
963 PyObject *v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
964
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
965 if (obj == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
966 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
967 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
968 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
969
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
970 #if 0
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
971 /* For b/w compatibility we also accept Unicode objects provided
20563
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
972 that no encodings is given and then redirect to
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
973 PyObject_Unicode() which then applies the additional logic for
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
974 Unicode subclasses.
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
975
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
976 NOTE: This API should really only be used for object which
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
977 represent *encoded* Unicode !
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
978
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
979 */
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
980 if (PyUnicode_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
981 if (encoding) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
982 PyErr_SetString(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
983 "decoding Unicode is not supported");
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
984 return NULL;
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
985 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
986 return PyObject_Unicode(obj);
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
987 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
988 #else
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
989 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
990 PyErr_SetString(PyExc_TypeError,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
991 "decoding Unicode is not supported");
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
992 return NULL;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
993 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
994 #endif
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
995
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
996 /* Coerce object */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
997 if (PyString_Check(obj)) {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
998 s = PyString_AS_STRING(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
999 len = PyString_GET_SIZE(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
1000 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1001 else if (PyObject_AsCharBuffer(obj, &s, &len)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1002 /* Overwrite the error message with something more useful in
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1003 case of a TypeError. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1004 if (PyErr_ExceptionMatches(PyExc_TypeError))
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
1005 PyErr_Format(PyExc_TypeError,
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1006 "coercing to Unicode: need string or buffer, "
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
1007 "%.80s found",
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
1008 obj->ob_type->tp_name);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1009 goto onError;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
1010 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1011
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1012 /* Convert to Unicode */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1013 if (len == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1014 Py_INCREF(unicode_empty);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1015 v = (PyObject *)unicode_empty;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1016 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1017 else
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1018 v = PyUnicode_Decode(s, len, encoding, errors);
15952
1f92d63b20f0 This patch adds a new builtin unistr() which behaves like str()
Marc-André Lemburg <mal@egenix.com>
parents: 15910
diff changeset
1019
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1020 return v;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1021
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1022 onError:
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
1023 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1024 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1025
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1026 PyObject *PyUnicode_Decode(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1027 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1028 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1029 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1030 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1031 PyObject *buffer = NULL, *unicode;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1032
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1033 if (encoding == NULL)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1034 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1035
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1036 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1037 if (strcmp(encoding, "utf-8") == 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1038 return PyUnicode_DecodeUTF8(s, size, errors);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1039 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1040 return PyUnicode_DecodeLatin1(s, size, errors);
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1041 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1042 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1043 return PyUnicode_DecodeMBCS(s, size, errors);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1044 #endif
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1045 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1046 return PyUnicode_DecodeASCII(s, size, errors);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1047
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1048 /* Decode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1049 buffer = PyBuffer_FromMemory((void *)s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1050 if (buffer == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1051 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1052 unicode = PyCodec_Decode(buffer, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1053 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1054 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1055 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1056 PyErr_Format(PyExc_TypeError,
12320
5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format
Guido van Rossum <guido@python.org>
parents: 12275
diff changeset
1057 "decoder did not return an unicode object (type=%.400s)",
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1058 unicode->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1059 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1060 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1061 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1062 Py_DECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1063 return unicode;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1064
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1065 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1066 Py_XDECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1067 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1068 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1069
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1070 PyObject *PyUnicode_AsDecodedObject(PyObject *unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1071 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1072 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1073 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1074 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1075
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1076 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1077 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1078 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1079 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1080
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1081 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1082 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1083
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1084 /* Decode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1085 v = PyCodec_Decode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1086 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1087 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1088 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1089
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1090 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1091 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1092 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1093
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1094 PyObject *PyUnicode_Encode(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1095 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1096 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1097 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1098 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1099 PyObject *v, *unicode;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1100
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1101 unicode = PyUnicode_FromUnicode(s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1102 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1103 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1104 v = PyUnicode_AsEncodedString(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1105 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1106 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1107 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1108
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1109 PyObject *PyUnicode_AsEncodedObject(PyObject *unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1110 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1111 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1112 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1113 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1114
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1115 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1116 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1117 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1118 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1119
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1120 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1121 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1122
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1123 /* Encode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1124 v = PyCodec_Encode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1125 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1126 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1127 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1128
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1129 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1130 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1131 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
1132
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1133 PyObject *PyUnicode_AsEncodedString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1134 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1135 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1136 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1137 PyObject *v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1138
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1139 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1140 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1141 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1142 }
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1143
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1144 if (encoding == NULL)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1145 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1146
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1147 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1148 if (errors == NULL) {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1149 if (strcmp(encoding, "utf-8") == 0)
17995
837825a779c1 fix bogus indentation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17887
diff changeset
1150 return PyUnicode_AsUTF8String(unicode);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1151 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1152 return PyUnicode_AsLatin1String(unicode);
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1153 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1154 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1155 return PyUnicode_AsMBCSString(unicode);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
1156 #endif
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1157 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1158 return PyUnicode_AsASCIIString(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1159 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1160
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1161 /* Encode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1162 v = PyCodec_Encode(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1163 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1164 goto onError;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1165 if (!PyBytes_Check(v)) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1166 if (PyString_Check(v)) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1167 /* Old codec, turn it into bytes */
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1168 PyObject *b = PyBytes_FromObject(v);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1169 Py_DECREF(v);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1170 return b;
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1171 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1172 PyErr_Format(PyExc_TypeError,
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1173 "encoder did not return a bytes object "
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1174 "(type=%.400s, encoding=%.20s, errors=%.20s)",
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1175 v->ob_type->tp_name,
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1176 encoding ? encoding : "NULL",
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1177 errors ? errors : "NULL");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1178 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1179 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1180 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1181 return v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1182
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1183 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1184 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1185 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1186
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1187 PyObject *_PyUnicode_AsDefaultEncodedString(PyObject *unicode,
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1188 const char *errors)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1189 {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1190 PyObject *v = ((PyUnicodeObject *)unicode)->defenc;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1191 PyObject *b;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1192 if (v)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1193 return v;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1194 if (errors != NULL)
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1195 Py_FatalError("non-NULL encoding in _PyUnicode_AsDefaultEncodedString");
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1196 if (errors == NULL) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1197 b = PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1198 PyUnicode_GET_SIZE(unicode),
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1199 NULL);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1200 }
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1201 else {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1202 b = PyUnicode_AsEncodedString(unicode, NULL, errors);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1203 }
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1204 if (!b)
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1205 return NULL;
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1206 v = PyString_FromStringAndSize(PyBytes_AsString(b),
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1207 PyBytes_Size(b));
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1208 Py_DECREF(b);
42208
ca3cfb34e3c2 Fixed a refcount leak in _PyUnicode_AsDefaultEncodedString().
Guido van Rossum <guido@python.org>
parents: 42184
diff changeset
1209 ((PyUnicodeObject *)unicode)->defenc = v;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1210 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1211 }
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
1212
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1213 char*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1214 PyUnicode_AsString(PyObject *unicode)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1215 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1216 assert(PyUnicode_Check(unicode));
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1217 unicode = _PyUnicode_AsDefaultEncodedString(unicode, NULL);
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1218 if (!unicode)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1219 return NULL;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1220 return PyString_AsString(unicode);
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1221 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
1222
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1223 Py_UNICODE *PyUnicode_AsUnicode(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1224 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1225 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1226 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1227 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1228 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1229 return PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1230
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1231 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1232 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1233 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1234
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1235 Py_ssize_t PyUnicode_GetSize(PyObject *unicode)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1236 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1237 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1238 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1239 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1240 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1241 return PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1242
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1243 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1244 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1245 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1246
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
1247 const char *PyUnicode_GetDefaultEncoding(void)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1248 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1249 return unicode_default_encoding;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1250 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1251
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1252 int PyUnicode_SetDefaultEncoding(const char *encoding)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1253 {
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1254 if (strcmp(encoding, unicode_default_encoding) != 0) {
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1255 PyErr_Format(PyExc_ValueError,
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1256 "Can only set default encoding to %s",
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1257 unicode_default_encoding);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1258 return -1;
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1259 }
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1260 return 0;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1261 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
1262
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1263 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1264 build arguments, call the callback and check the arguments,
35806
44fe157255bc fix typos, mostly in comments
Fred Drake <fdrake@acm.org>
parents: 35758
diff changeset
1265 if no exception occurred, copy the replacement to the output
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1266 and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1267 return 0 on success, -1 on error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1268 */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1269
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1270 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1271 int unicode_decode_call_errorhandler(const char *errors, PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1272 const char *encoding, const char *reason,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1273 const char *input, Py_ssize_t insize, Py_ssize_t *startinpos, Py_ssize_t *endinpos, PyObject **exceptionObject, const char **inptr,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1274 PyObject **output, Py_ssize_t *outpos, Py_UNICODE **outptr)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1275 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1276 static char *argparse = "O!n;decoding error handler must return (unicode, int) tuple";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1277
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1278 PyObject *restuple = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1279 PyObject *repunicode = NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1280 Py_ssize_t outsize = PyUnicode_GET_SIZE(*output);
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1281 Py_ssize_t requiredsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1282 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1283 Py_UNICODE *repptr;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1284 Py_ssize_t repsize;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1285 int res = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1286
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1287 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1288 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1289 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1290 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1291 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1292
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1293 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1294 *exceptionObject = PyUnicodeDecodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1295 encoding, input, insize, *startinpos, *endinpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1296 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1297 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1298 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1299 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1300 if (PyUnicodeDecodeError_SetStart(*exceptionObject, *startinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1301 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1302 if (PyUnicodeDecodeError_SetEnd(*exceptionObject, *endinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1303 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1304 if (PyUnicodeDecodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1305 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1306 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1307
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1308 restuple = PyObject_CallFunctionObjArgs(*errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1309 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1310 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1311 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1312 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1313 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1314 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1315 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type, &repunicode, &newpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1316 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1317 if (newpos<0)
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1318 newpos = insize+newpos;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1319 if (newpos<0 || newpos>insize) {
36510
0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat.
Martin v. Löwis <martin@v.loewis.de>
parents: 36508
diff changeset
1320 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", newpos);
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1321 goto onError;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
1322 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1323
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1324 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1325 have+the replacement+the rest of the string (starting
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1326 at the new input position), so we won't have to check space
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1327 when there are no errors in the rest of the string) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1328 repptr = PyUnicode_AS_UNICODE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1329 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1330 requiredsize = *outpos + repsize + insize-newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1331 if (requiredsize > outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1332 if (requiredsize<2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1333 requiredsize = 2*outsize;
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1334 if (PyUnicode_Resize(output, requiredsize) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1335 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1336 *outptr = PyUnicode_AS_UNICODE(*output) + *outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1337 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1338 *endinpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1339 *inptr = input + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1340 Py_UNICODE_COPY(*outptr, repptr, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1341 *outptr += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1342 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1343 /* we made it! */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1344 res = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1345
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1346 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1347 Py_XDECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1348 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1349 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1350
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1351 /* --- UTF-7 Codec -------------------------------------------------------- */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1352
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1353 /* see RFC2152 for details */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1354
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1355 static
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1356 char utf7_special[128] = {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1357 /* indicate whether a UTF-7 character is special i.e. cannot be directly
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1358 encoded:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1359 0 - not special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1360 1 - special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1361 2 - whitespace (optional)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1362 3 - RFC2152 Set O (optional) */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1363 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1364 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1365 2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1366 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1367 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1368 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1369 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1370 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1371
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1372 };
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1373
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1374 /* Note: The comparison (c) <= 0 is a trick to work-around gcc
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1375 warnings about the comparison always being false; since
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1376 utf7_special[0] is 1, we can safely make that one comparison
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1377 true */
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1378
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1379 #define SPECIAL(c, encodeO, encodeWS) \
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1380 ((c) > 127 || (c) <= 0 || utf7_special[(c)] == 1 || \
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1381 (encodeWS && (utf7_special[(c)] == 2)) || \
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1382 (encodeO && (utf7_special[(c)] == 3)))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1383
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1384 #define B64(n) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1385 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[(n) & 0x3f])
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1386 #define B64CHAR(c) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1387 (isalnum(c) || (c) == '+' || (c) == '/')
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1388 #define UB64(c) \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1389 ((c) == '+' ? 62 : (c) == '/' ? 63 : (c) >= 'a' ? \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1390 (c) - 71 : (c) >= 'A' ? (c) - 65 : (c) + 4 )
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1391
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1392 #define ENCODE(out, ch, bits) \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1393 while (bits >= 6) { \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1394 *out++ = B64(ch >> (bits-6)); \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1395 bits -= 6; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1396 }
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1397
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1398 #define DECODE(out, ch, bits, surrogate) \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1399 while (bits >= 16) { \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1400 Py_UNICODE outCh = (Py_UNICODE) ((ch >> (bits-16)) & 0xffff); \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1401 bits -= 16; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1402 if (surrogate) { \
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1403 /* We have already generated an error for the high surrogate \
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1404 so let's not bother seeing if the low surrogate is correct or not */ \
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1405 surrogate = 0; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1406 } else if (0xDC00 <= outCh && outCh <= 0xDFFF) { \
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1407 /* This is a surrogate pair. Unfortunately we can't represent \
35729
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1408 it in a 16-bit character */ \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1409 surrogate = 1; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1410 errmsg = "code pairs are not supported"; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1411 goto utf7Error; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1412 } else { \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1413 *out++ = outCh; \
6913abb7047a Whitespace corrections.
Marc-André Lemburg <mal@egenix.com>
parents: 35728
diff changeset
1414 } \
35728
9ed43e637ae7 Bug fix for [ 1331062 ] utf 7 codec broken.
Marc-André Lemburg <mal@egenix.com>
parents: 35702
diff changeset
1415 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1416
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1417 PyObject *PyUnicode_DecodeUTF7(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1418 Py_ssize_t size,
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1419 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1420 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1421 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1422 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1423 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1424 Py_ssize_t outpos;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1425 const char *e;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1426 PyUnicodeObject *unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1427 Py_UNICODE *p;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1428 const char *errmsg = "";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1429 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1430 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1431 unsigned long charsleft = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1432 int surrogate = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1433 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1434 PyObject *exc = NULL;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1435
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1436 unicode = _PyUnicode_New(size);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1437 if (!unicode)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1438 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1439 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1440 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1441
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1442 p = unicode->str;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1443 e = s + size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1444
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1445 while (s < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1446 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1447 restart:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1448 ch = *s;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1449
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1450 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1451 if ((ch == '-') || !B64CHAR(ch)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1452 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1453 s++;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1454
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1455 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1456 if (bitsleft >= 6) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1457 /* The shift sequence has a partial character in it. If
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1458 bitsleft < 6 then we could just classify it as padding
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1459 but that is not the case here */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1460
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1461 errmsg = "partial character in shift sequence";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1462 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1463 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1464 /* According to RFC2152 the remaining bits should be zero. We
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1465 choose to signal an error/insert a replacement character
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1466 here so indicate the potential of a misencoded character. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1467
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1468 /* On x86, a << b == a << (b%32) so make sure that bitsleft != 0 */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1469 if (bitsleft && charsleft << (sizeof(charsleft) * 8 - bitsleft)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1470 errmsg = "non-zero padding bits in shift sequence";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1471 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1472 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1473
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1474 if (ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1475 if ((s < e) && (*(s) == '-')) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1476 *p++ = '-';
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1477 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1478 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1479 } else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1480 errmsg = "unexpected special character";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1481 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1482 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1483 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1484 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1485 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1486 charsleft = (charsleft << 6) | UB64(ch);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1487 bitsleft += 6;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1488 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1489 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1490 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1491 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1492 else if ( ch == '+' ) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1493 startinpos = s-starts;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1494 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1495 if (s < e && *s == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1496 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1497 *p++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1498 } else
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1499 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1500 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1501 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1502 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1503 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1504 else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1505 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1506 s++;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1507 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1508 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1509 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1510 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1511 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1512 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1513 continue;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1514 utf7Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1515 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1516 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1517 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1518 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1519 "utf7", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1520 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1521 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1522 goto onError;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1523 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1524
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1525 if (inShift) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1526 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1527 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1528 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1529 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1530 "utf7", "unterminated shift sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1531 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1532 (PyObject **)&unicode, &outpos, &p))
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1533 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1534 if (s < e)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1535 goto restart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1536 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1537
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1538 if (_PyUnicode_Resize(&unicode, p - PyUnicode_AS_UNICODE(unicode)) < 0)
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1539 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1540
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1541 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1542 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1543 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1544
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1545 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1546 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1547 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1548 Py_DECREF(unicode);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1549 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1550 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1551
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1552
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1553 PyObject *PyUnicode_EncodeUTF7(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1554 Py_ssize_t size,
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1555 int encodeSetO,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1556 int encodeWhiteSpace,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1557 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1558 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1559 PyObject *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1560 /* It might be possible to tighten this worst case */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1561 Py_ssize_t cbAllocated = 5 * size;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1562 int inShift = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1563 Py_ssize_t i = 0;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1564 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1565 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1566 char * out;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1567 char * start;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1568
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1569 if (size == 0)
41624
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1570 return PyBytes_FromStringAndSize(NULL, 0);
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1571
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1572 v = PyBytes_FromStringAndSize(NULL, cbAllocated);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1573 if (v == NULL)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1574 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1575
41624
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1576 start = out = PyBytes_AS_STRING(v);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1577 for (;i < size; ++i) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1578 Py_UNICODE ch = s[i];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1579
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1580 if (!inShift) {
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1581 if (ch == '+') {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1582 *out++ = '+';
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1583 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1584 } else if (SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1585 charsleft = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1586 bitsleft = 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1587 *out++ = '+';
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1588 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1589 inShift = bitsleft > 0;
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1590 } else {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1591 *out++ = (char) ch;
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1592 }
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1593 } else {
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1594 if (!SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1595 *out++ = B64(charsleft << (6-bitsleft));
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1596 charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1597 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1598 /* Characters not in the BASE64 set implicitly unshift the sequence
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1599 so no '-' is required, except if the character is itself a '-' */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1600 if (B64CHAR(ch) || ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1601 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1602 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1603 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1604 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1605 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1606 bitsleft += 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1607 charsleft = (charsleft << 16) | ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1608 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1609
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1610 /* If the next character is special then we dont' need to terminate
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1611 the shift sequence. If the next character is not a BASE64 character
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1612 or '-' then the shift sequence will be terminated implicitly and we
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1613 don't have to insert a '-'. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1614
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1615 if (bitsleft == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1616 if (i + 1 < size) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1617 Py_UNICODE ch2 = s[i+1];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1618
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1619 if (SPECIAL(ch2, encodeSetO, encodeWhiteSpace)) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1620
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1621 } else if (B64CHAR(ch2) || ch2 == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1622 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1623 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1624 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1625 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1626 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1627
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1628 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1629 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1630 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1631 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1632 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1633 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1634 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1635 }
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1636 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1637 if (bitsleft) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1638 *out++= B64(charsleft << (6-bitsleft) );
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1639 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1640 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1641
41624
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1642 if (PyBytes_Resize(v, out - start)) {
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1643 Py_DECREF(v);
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1644 return NULL;
e25dcd874b82 Change PyUnicode_EncodeUTF7() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41623
diff changeset
1645 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1646 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1647 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1648
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1649 #undef SPECIAL
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1650 #undef B64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1651 #undef B64CHAR
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1652 #undef UB64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1653 #undef ENCODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1654 #undef DECODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1655
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1656 /* --- UTF-8 Codec -------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1657
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1658 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1659 char utf8_code_length[256] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1660 /* Map UTF-8 encoded prefix byte to sequence length. zero means
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1661 illegal prefix. see RFC 2279 for details */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1662 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1663 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1664 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1665 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1666 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1667 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1668 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1669 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1670 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1671 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1672 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1673 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1674 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1675 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1676 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1677 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1678 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1679
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1680 PyObject *PyUnicode_DecodeUTF8(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1681 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1682 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1683 {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1684 return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1685 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1686
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1687 PyObject *PyUnicode_DecodeUTF8Stateful(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1688 Py_ssize_t size,
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1689 const char *errors,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1690 Py_ssize_t *consumed)
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1691 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1692 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1693 int n;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1694 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1695 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1696 Py_ssize_t outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1697 const char *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1698 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1699 Py_UNICODE *p;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1700 const char *errmsg = "";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1701 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1702 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1703
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1704 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1705 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1706 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1707 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1708 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1709 if (size == 0) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1710 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1711 *consumed = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1712 return (PyObject *)unicode;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1713 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1714
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1715 /* Unpack UTF-8 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1716 p = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1717 e = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1718
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1719 while (s < e) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1720 Py_UCS4 ch = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1721
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1722 if (ch < 0x80) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1723 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1724 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1725 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1726 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1727
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1728 n = utf8_code_length[ch];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1729
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1730 if (s + n > e) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1731 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1732 break;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1733 else {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1734 errmsg = "unexpected end of data";
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1735 startinpos = s-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1736 endinpos = size;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1737 goto utf8Error;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1738 }
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1739 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1740
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1741 switch (n) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1742
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1743 case 0:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1744 errmsg = "unexpected code byte";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1745 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1746 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1747 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1748
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1749 case 1:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1750 errmsg = "internal error";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1751 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1752 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1753 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1754
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1755 case 2:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1756 if ((s[1] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1757 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1758 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1759 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1760 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1761 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1762 ch = ((s[0] & 0x1f) << 6) + (s[1] & 0x3f);
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1763 if (ch < 0x80) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1764 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1765 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1766 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1767 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1768 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1769 else
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1770 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1771 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1772
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1773 case 3:
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1774 if ((s[1] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1775 (s[2] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1776 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1777 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1778 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1779 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1780 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1781 ch = ((s[0] & 0x0f) << 12) + ((s[1] & 0x3f) << 6) + (s[2] & 0x3f);
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1782 if (ch < 0x0800) {
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1783 /* Note: UTF-8 encodings of surrogates are considered
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1784 legal UTF-8 sequences;
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1785
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1786 XXX For wide builds (UCS-4) we should probably try
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1787 to recombine the surrogates into a single code
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1788 unit.
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1789 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1790 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1791 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1792 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1793 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1794 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1795 else
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1796 *p++ = (Py_UNICODE)ch;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1797 break;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1798
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1799 case 4:
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1800 if ((s[1] & 0xc0) != 0x80 ||
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1801 (s[2] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1802 (s[3] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1803 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1804 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1805 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1806 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1807 }
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1808 ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1809 ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1810 /* validate and convert to UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1811 if ((ch < 0x10000) /* minimum value allowed for 4
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1812 byte encoding */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1813 || (ch > 0x10ffff)) /* maximum value allowed for
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1814 UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1815 {
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1816 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1817 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1818 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1819 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1820 }
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1821 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1822 *p++ = (Py_UNICODE)ch;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1823 #else
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1824 /* compute and append the two surrogates: */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1825
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1826 /* translate from 10000..10FFFF to 0..FFFF */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1827 ch -= 0x10000;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1828
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1829 /* high surrogate = top 10 bits added to D800 */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1830 *p++ = (Py_UNICODE)(0xD800 + (ch >> 10));
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1831
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1832 /* low surrogate = bottom 10 bits added to DC00 */
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
1833 *p++ = (Py_UNICODE)(0xDC00 + (ch & 0x03FF));
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1834 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1835 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1836
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1837 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1838 /* Other sizes are only needed for UCS-4 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1839 errmsg = "unsupported Unicode code range";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1840 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1841 endinpos = startinpos+n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1842 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1843 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1844 s += n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1845 continue;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1846
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1847 utf8Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1848 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1849 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1850 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1851 "utf8", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1852 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1853 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1854 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1855 }
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1856 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1857 *consumed = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1858
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1859 /* Adjust length */
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1860 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1861 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1862
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1863 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1864 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1865 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1866
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1867 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1868 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1869 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1870 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1871 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1872 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1873
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1874 /* Allocation strategy: if the string is short, convert into a stack buffer
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1875 and allocate exactly as much space needed at the end. Else allocate the
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1876 maximum possible needed (4 result bytes per Unicode character), and return
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1877 the excess memory at the end.
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1878 */
23198
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1879 PyObject *
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1880 PyUnicode_EncodeUTF8(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1881 Py_ssize_t size,
23198
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1882 const char *errors)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1883 {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1884 #define MAX_SHORT_UNICHARS 300 /* largest size we'll do on the stack */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1885
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1886 Py_ssize_t i; /* index into s of next input byte */
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1887 PyObject *v; /* result string object */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1888 char *p; /* next free byte in output buffer */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1889 Py_ssize_t nallocated; /* number of result bytes allocated */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
1890 Py_ssize_t nneeded; /* number of result bytes needed */
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1891 char stackbuf[MAX_SHORT_UNICHARS * 4];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1892
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1893 assert(s != NULL);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1894 assert(size >= 0);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1895
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1896 if (size <= MAX_SHORT_UNICHARS) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1897 /* Write into the stack buffer; nallocated can't overflow.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1898 * At the end, we'll allocate exactly as much heap space as it
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1899 * turns out we need.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1900 */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1901 nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1902 v = NULL; /* will allocate after we're done */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1903 p = stackbuf;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1904 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1905 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1906 /* Overallocate on the heap, and give the excess back at the end. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1907 nallocated = size * 4;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1908 if (nallocated / 4 != size) /* overflow! */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1909 return PyErr_NoMemory();
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1910 v = PyBytes_FromStringAndSize(NULL, nallocated);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1911 if (v == NULL)
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1912 return NULL;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1913 p = PyBytes_AS_STRING(v);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1914 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1915
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1916 for (i = 0; i < size;) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1917 Py_UCS4 ch = s[i++];
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1918
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1919 if (ch < 0x80)
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1920 /* Encode ASCII */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1921 *p++ = (char) ch;
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1922
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1923 else if (ch < 0x0800) {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1924 /* Encode Latin-1 */
21884
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1925 *p++ = (char)(0xc0 | (ch >> 6));
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1926 *p++ = (char)(0x80 | (ch & 0x3f));
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1927 }
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1928 else {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1929 /* Encode UCS2 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1930 if (ch < 0x10000) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1931 /* Special case: check for high surrogate */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1932 if (0xD800 <= ch && ch <= 0xDBFF && i != size) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1933 Py_UCS4 ch2 = s[i];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1934 /* Check for low surrogate and combine the two to
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1935 form a UCS4 value */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1936 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1937 ch = ((ch - 0xD800) << 10 | (ch2 - 0xDC00)) + 0x10000;
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1938 i++;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1939 goto encodeUCS4;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1940 }
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1941 /* Fall through: handles isolated high surrogates */
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1942 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1943 *p++ = (char)(0xe0 | (ch >> 12));
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1944 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1945 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1946 continue;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1947 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1948 encodeUCS4:
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1949 /* Encode UCS4 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1950 *p++ = (char)(0xf0 | (ch >> 18));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1951 *p++ = (char)(0x80 | ((ch >> 12) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1952 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1953 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1954 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1955 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1956
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1957 if (v == NULL) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1958 /* This was stack allocated. */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
1959 nneeded = p - stackbuf;
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1960 assert(nneeded <= nallocated);
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1961 v = PyBytes_FromStringAndSize(stackbuf, nneeded);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1962 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1963 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1964 /* Cut back to size actually needed. */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1965 nneeded = p - PyBytes_AS_STRING(v);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1966 assert(nneeded <= nallocated);
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
1967 PyBytes_Resize(v, nneeded);
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1968 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1969 return v;
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1970
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1971 #undef MAX_SHORT_UNICHARS
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1972 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1973
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1974 PyObject *PyUnicode_AsUTF8String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1975 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1976 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1977 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1978 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1979 }
14059
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1980 return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1981 PyUnicode_GET_SIZE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1982 NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1983 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1984
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1985 /* --- UTF-16 Codec ------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1986
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1987 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1988 PyUnicode_DecodeUTF16(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1989 Py_ssize_t size,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1990 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1991 int *byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1992 {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1993 return PyUnicode_DecodeUTF16Stateful(s, size, errors, byteorder, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1994 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1995
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1996 PyObject *
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1997 PyUnicode_DecodeUTF16Stateful(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
1998 Py_ssize_t size,
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1999 const char *errors,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2000 int *byteorder,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2001 Py_ssize_t *consumed)
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2002 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2003 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2004 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2005 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2006 Py_ssize_t outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2007 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2008 Py_UNICODE *p;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2009 const unsigned char *q, *e;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2010 int bo = 0; /* assume native ordering by default */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2011 const char *errmsg = "";
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2012 /* Offsets from q for retrieving byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2013 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2014 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2015 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2016 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2017 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2018 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2019 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2020
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2021 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2022 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2023 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2024 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2025 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2026 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2027 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2028
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2029 /* Unpack UTF-16 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2030 p = unicode->str;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2031 q = (unsigned char *)s;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2032 e = q + size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2033
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2034 if (byteorder)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2035 bo = *byteorder;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2036
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2037 /* Check for BOM marks (U+FEFF) in the input and adjust current
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2038 byte order setting accordingly. In native mode, the leading BOM
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2039 mark is skipped, in all other modes, it is copied to the output
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2040 stream as-is (giving a ZWNBSP character). */
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2041 if (bo == 0) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2042 if (size >= 2) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2043 const Py_UNICODE bom = (q[ihi] << 8) | q[ilo];
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2044 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2045 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2046 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2047 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2048 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2049 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2050 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2051 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2052 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2053 #else
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2054 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2055 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2056 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2057 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2058 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2059 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2060 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2061 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2062 #endif
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2063 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
2064 }
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2065
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2066 if (bo == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2067 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2068 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2069 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2070 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2071 else if (bo == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2072 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2073 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2074 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2075 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2076
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2077 while (q < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2078 Py_UNICODE ch;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2079 /* remaining bytes at the end? (size should be even) */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2080 if (e-q<2) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2081 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2082 break;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2083 errmsg = "truncated data";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2084 startinpos = ((const char *)q)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2085 endinpos = ((const char *)e)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2086 goto utf16Error;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2087 /* The remaining input chars are ignored if the callback
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2088 chooses to skip the input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2089 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2090 ch = (q[ihi] << 8) | q[ilo];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2091
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2092 q += 2;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2093
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2094 if (ch < 0xD800 || ch > 0xDFFF) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2095 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2096 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2097 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2098
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2099 /* UTF-16 code pair: */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2100 if (q >= e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2101 errmsg = "unexpected end of data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2102 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2103 endinpos = ((const char *)e)-starts;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2104 goto utf16Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2105 }
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
2106 if (0xD800 <= ch && ch <= 0xDBFF) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2107 Py_UNICODE ch2 = (q[ihi] << 8) | q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2108 q += 2;
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
2109 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
2110 #ifndef Py_UNICODE_WIDE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2111 *p++ = ch;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2112 *p++ = ch2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2113 #else
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2114 *p++ = (((ch & 0x3FF)<<10) | (ch2 & 0x3FF)) + 0x10000;
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2115 #endif
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2116 continue;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2117 }
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2118 else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2119 errmsg = "illegal UTF-16 surrogate";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2120 startinpos = (((const char *)q)-4)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2121 endinpos = startinpos+2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2122 goto utf16Error;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2123 }
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2124
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2125 }
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2126 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2127 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2128 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2129 /* Fall through to report the error */
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2130
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2131 utf16Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2132 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2133 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2134 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2135 "utf16", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2136 starts, size, &startinpos, &endinpos, &exc, (const char **)&q,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2137 (PyObject **)&unicode, &outpos, &p))
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
2138 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2139 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2140
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2141 if (byteorder)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2142 *byteorder = bo;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2143
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2144 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2145 *consumed = (const char *)q-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
2146
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2147 /* Adjust length */
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
2148 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2149 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2150
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2151 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2152 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2153 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2154
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2155 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2156 Py_DECREF(unicode);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2157 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2158 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2159 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2160 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2161
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2162 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2163 PyUnicode_EncodeUTF16(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2164 Py_ssize_t size,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2165 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2166 int byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2167 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2168 PyObject *v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2169 unsigned char *p;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2170 #ifdef Py_UNICODE_WIDE
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2171 int i, pairs;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2172 #else
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2173 const int pairs = 0;
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2174 #endif
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2175 /* Offsets from p for storing byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2176 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2177 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2178 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2179 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2180 #endif
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2181
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2182 #define STORECHAR(CH) \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2183 do { \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2184 p[ihi] = ((CH) >> 8) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2185 p[ilo] = (CH) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2186 p += 2; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2187 } while(0)
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2188
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2189 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2190 for (i = pairs = 0; i < size; i++)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2191 if (s[i] >= 0x10000)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2192 pairs++;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2193 #endif
41605
1162502a09da Change PyUnicode_EncodeUTF16() so that it returns
Walter Dörwald <walter@livinglogic.de>
parents: 41598
diff changeset
2194 v = PyBytes_FromStringAndSize(NULL,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2195 2 * (size + pairs + (byteorder == 0)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2196 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2197 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2198
41605
1162502a09da Change PyUnicode_EncodeUTF16() so that it returns
Walter Dörwald <walter@livinglogic.de>
parents: 41598
diff changeset
2199 p = (unsigned char *)PyBytes_AS_STRING(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2200 if (byteorder == 0)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2201 STORECHAR(0xFEFF);
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
2202 if (size == 0)
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2203 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2204
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2205 if (byteorder == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2206 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2207 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2208 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2209 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2210 else if (byteorder == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2211 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2212 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2213 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2214 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2215
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2216 while (size-- > 0) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2217 Py_UNICODE ch = *s++;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2218 Py_UNICODE ch2 = 0;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2219 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2220 if (ch >= 0x10000) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2221 ch2 = 0xDC00 | ((ch-0x10000) & 0x3FF);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2222 ch = 0xD800 | ((ch-0x10000) >> 10);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2223 }
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2224 #endif
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2225 STORECHAR(ch);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2226 if (ch2)
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2227 STORECHAR(ch2);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2228 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2229 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
2230 #undef STORECHAR
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2231 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2232
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2233 PyObject *PyUnicode_AsUTF16String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2234 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2235 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2236 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2237 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2238 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2239 return PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2240 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2241 NULL,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2242 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2243 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2244
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2245 /* --- Unicode Escape Codec ----------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2246
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2247 static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL;
13063
eb243443b884 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13049
diff changeset
2248
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2249 PyObject *PyUnicode_DecodeUnicodeEscape(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2250 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2251 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2252 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2253 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2254 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2255 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2256 Py_ssize_t outpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2257 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2258 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2259 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2260 const char *end;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2261 char* message;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2262 Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2263 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2264 PyObject *exc = NULL;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2265
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2266 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2267 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2268 length after conversion to the true value.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2269 (but if the error callback returns a long replacement string
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2270 we'll have to allocate more space) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2271 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2272 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2273 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2274 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2275 return (PyObject *)v;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2276
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2277 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2278 end = s + size;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2279
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2280 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2281 unsigned char c;
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
2282 Py_UNICODE x;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2283 int digits;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2284
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2285 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2286 if (*s != '\\') {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2287 *p++ = (unsigned char) *s++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2288 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2289 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2290
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2291 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2292 /* \ - Escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2293 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2294 switch (*s++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2295
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2296 /* \x escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2297 case '\n': break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2298 case '\\': *p++ = '\\'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2299 case '\'': *p++ = '\''; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2300 case '\"': *p++ = '\"'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2301 case 'b': *p++ = '\b'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2302 case 'f': *p++ = '\014'; break; /* FF */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2303 case 't': *p++ = '\t'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2304 case 'n': *p++ = '\n'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2305 case 'r': *p++ = '\r'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2306 case 'v': *p++ = '\013'; break; /* VT */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2307 case 'a': *p++ = '\007'; break; /* BEL, not classic C */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2308
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2309 /* \OOO (octal) escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2310 case '0': case '1': case '2': case '3':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2311 case '4': case '5': case '6': case '7':
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2312 x = s[-1] - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2313 if ('0' <= *s && *s <= '7') {
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2314 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2315 if ('0' <= *s && *s <= '7')
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2316 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2317 }
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
2318 *p++ = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2319 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2320
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2321 /* hex escapes */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2322 /* \xXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2323 case 'x':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2324 digits = 2;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2325 message = "truncated \\xXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2326 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2327
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2328 /* \uXXXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2329 case 'u':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2330 digits = 4;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2331 message = "truncated \\uXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2332 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2333
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2334 /* \UXXXXXXXX */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2335 case 'U':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2336 digits = 8;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2337 message = "truncated \\UXXXXXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2338 hexescape:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2339 chr = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2340 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2341 if (s+digits>end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2342 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2343 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2344 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2345 "unicodeescape", "end of string in escape sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2346 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2347 (PyObject **)&v, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2348 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2349 goto nextByte;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2350 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2351 for (i = 0; i < digits; ++i) {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2352 c = (unsigned char) s[i];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2353 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2354 endinpos = (s+i+1)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2355 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2356 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2357 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2358 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2359 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2360 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2361 goto nextByte;
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2362 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2363 chr = (chr<<4) & ~0xF;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2364 if (c >= '0' && c <= '9')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2365 chr += c - '0';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2366 else if (c >= 'a' && c <= 'f')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2367 chr += 10 + c - 'a';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2368 else
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2369 chr += 10 + c - 'A';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2370 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2371 s += i;
30653
3b47d6057151 Fix for SF bug [ 817156 ] invalid \U escape gives 0=length unistr.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30441
diff changeset
2372 if (chr == 0xffffffff && PyErr_Occurred())
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2373 /* _decoding_error will have already written into the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2374 target buffer. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2375 break;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2376 store:
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2377 /* when we get here, chr is a 32-bit unicode character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2378 if (chr <= 0xffff)
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2379 /* UCS-2 character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2380 *p++ = (Py_UNICODE) chr;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2381 else if (chr <= 0x10ffff) {
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2382 /* UCS-4 character. Either store directly, or as
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2383 surrogate pair. */
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
2384 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2385 *p++ = chr;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2386 #else
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2387 chr -= 0x10000L;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2388 *p++ = 0xD800 + (Py_UNICODE) (chr >> 10);
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
2389 *p++ = 0xDC00 + (Py_UNICODE) (chr & 0x03FF);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2390 #endif
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2391 } else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2392 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2393 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2394 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2395 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2396 "unicodeescape", "illegal Unicode character",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2397 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2398 (PyObject **)&v, &outpos, &p))
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2399 goto onError;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
2400 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2401 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2402
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2403 /* \N{name} */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2404 case 'N':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2405 message = "malformed \\N character escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2406 if (ucnhash_CAPI == NULL) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2407 /* load the unicode data module */
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2408 PyObject *m, *api;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2409 m = PyImport_ImportModule("unicodedata");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2410 if (m == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2411 goto ucnhashError;
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2412 api = PyObject_GetAttrString(m, "ucnhash_CAPI");
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2413 Py_DECREF(m);
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2414 if (api == NULL)
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2415 goto ucnhashError;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2416 ucnhash_CAPI = (_PyUnicode_Name_CAPI *)PyCObject_AsVoidPtr(api);
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2417 Py_DECREF(api);
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2418 if (ucnhash_CAPI == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2419 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2420 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2421 if (*s == '{') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2422 const char *start = s+1;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2423 /* look for the closing brace */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2424 while (*s != '}' && s < end)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2425 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2426 if (s > start && s < end && *s == '}') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2427 /* found a name. look it up in the unicode database */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2428 message = "unknown Unicode character name";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2429 s++;
36924
48e0a5e39403 Update Unicode database to Unicode 4.1.
Martin v. Löwis <martin@v.loewis.de>
parents: 36882
diff changeset
2430 if (ucnhash_CAPI->getcode(NULL, start, (int)(s-start-1), &chr))
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2431 goto store;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2432 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2433 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2434 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2435 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2436 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2437 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2438 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2439 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2440 (PyObject **)&v, &outpos, &p))
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2441 goto onError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2442 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2443
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2444 default:
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2445 if (s > end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2446 message = "\\ at end of string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2447 s--;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2448 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2449 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2450 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2451 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2452 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2453 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2454 (PyObject **)&v, &outpos, &p))
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2455 goto onError;
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2456 }
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2457 else {
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2458 *p++ = '\\';
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2459 *p++ = (unsigned char)s[-1];
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2460 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2461 break;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2462 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2463 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2464 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2465 }
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2466 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2467 goto onError;
30160
3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()).
Walter Dörwald <walter@livinglogic.de>
parents: 30149
diff changeset
2468 Py_XDECREF(errorHandler);
3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()).
Walter Dörwald <walter@livinglogic.de>
parents: 30149
diff changeset
2469 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2470 return (PyObject *)v;
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
2471
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2472 ucnhashError:
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2473 PyErr_SetString(
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2474 PyExc_UnicodeError,
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2475 "\\N escapes not supported (can't load unicodedata module)"
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
2476 );
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
2477 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2478 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2479 Py_XDECREF(exc);
16064
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
2480 return NULL;
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
2481
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
2482 onError:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2483 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2484 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2485 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2486 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2487 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2488
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2489 /* Return a Unicode-Escape string version of the Unicode object.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2490
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2491 If quotes is true, the string is enclosed in u"" or u'' quotes as
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2492 appropriate.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2493
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2494 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2495
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2496 Py_LOCAL_INLINE(const Py_UNICODE *) findchar(const Py_UNICODE *s,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2497 Py_ssize_t size,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2498 Py_UNICODE ch)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2499 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2500 /* like wcschr, but doesn't stop at NULL characters */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2501
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2502 while (size-- > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2503 if (*s == ch)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2504 return s;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2505 s++;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2506 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2507
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2508 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2509 }
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
2510
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2511 static const char *hexdigits = "0123456789abcdef";
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2512
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2513 PyObject *PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s,
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2514 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2515 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2516 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2517 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2518
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2519 /* XXX(nnorwitz): rather than over-allocating, it would be
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2520 better to choose a different scheme. Perhaps scan the
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2521 first N-chars of the string and allocate based on that size.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2522 */
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2523 /* Initial allocation is based on the longest-possible unichr
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2524 escape.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2525
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2526 In wide (UTF-32) builds '\U00xxxxxx' is 10 chars per source
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2527 unichr, so in this case it's the longest unichr escape. In
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2528 narrow (UTF-16) builds this is five chars per source unichr
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2529 since there are two unichrs in the surrogate pair, so in narrow
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2530 (UTF-16) builds it's not the longest unichr escape.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2531
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2532 In wide or narrow builds '\uxxxx' is 6 chars per source unichr,
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2533 so in the narrow (UTF-16) build case it's the longest unichr
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2534 escape.
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2535 */
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2536
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2537 repr = PyBytes_FromStringAndSize(NULL,
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2538 #ifdef Py_UNICODE_WIDE
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2539 + 10*size
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2540 #else
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2541 + 6*size
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2542 #endif
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2543 + 1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2544 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2545 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2546
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2547 p = PyBytes_AS_STRING(repr);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2548
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2549 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2550 Py_UNICODE ch = *s++;
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2551
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2552 /* Escape backslashes */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2553 if (ch == '\\') {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2554 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2555 *p++ = (char) ch;
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2556 continue;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2557 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2558
18773
743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This
Guido van Rossum <guido@python.org>
parents: 18324
diff changeset
2559 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2560 /* Map 21-bit characters to '\U00xxxxxx' */
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2561 else if (ch >= 0x10000) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2562 *p++ = '\\';
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2563 *p++ = 'U';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2564 *p++ = hexdigits[(ch >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2565 *p++ = hexdigits[(ch >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2566 *p++ = hexdigits[(ch >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2567 *p++ = hexdigits[(ch >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2568 *p++ = hexdigits[(ch >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2569 *p++ = hexdigits[(ch >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2570 *p++ = hexdigits[(ch >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2571 *p++ = hexdigits[ch & 0x0000000F];
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2572 continue;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2573 }
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2574 #else
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2575 /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2576 else if (ch >= 0xD800 && ch < 0xDC00) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2577 Py_UNICODE ch2;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2578 Py_UCS4 ucs;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2579
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2580 ch2 = *s++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2581 size--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2582 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2583 ucs = (((ch & 0x03FF) << 10) | (ch2 & 0x03FF)) + 0x00010000;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2584 *p++ = '\\';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2585 *p++ = 'U';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2586 *p++ = hexdigits[(ucs >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2587 *p++ = hexdigits[(ucs >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2588 *p++ = hexdigits[(ucs >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2589 *p++ = hexdigits[(ucs >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2590 *p++ = hexdigits[(ucs >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2591 *p++ = hexdigits[(ucs >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2592 *p++ = hexdigits[(ucs >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2593 *p++ = hexdigits[ucs & 0x0000000F];
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2594 continue;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2595 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2596 /* Fall through: isolated surrogates are copied as-is */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2597 s--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2598 size++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2599 }
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2600 #endif
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2601
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2602 /* Map 16-bit characters to '\uxxxx' */
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2603 if (ch >= 256) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2604 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2605 *p++ = 'u';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2606 *p++ = hexdigits[(ch >> 12) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2607 *p++ = hexdigits[(ch >> 8) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2608 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2609 *p++ = hexdigits[ch & 0x000F];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2610 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2611
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2612 /* Map special whitespace to '\t', \n', '\r' */
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2613 else if (ch == '\t') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2614 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2615 *p++ = 't';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2616 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2617 else if (ch == '\n') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2618 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2619 *p++ = 'n';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2620 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2621 else if (ch == '\r') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2622 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2623 *p++ = 'r';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2624 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2625
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2626 /* Map non-printable US ASCII to '\xhh' */
21067
c9c4c523dcbf Fix for bug #485951: repr diff between string and unicode.
Marc-André Lemburg <mal@egenix.com>
parents: 21028
diff changeset
2627 else if (ch < ' ' || ch >= 0x7F) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2628 *p++ = '\\';
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2629 *p++ = 'x';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2630 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2631 *p++ = hexdigits[ch & 0x000F];
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2632 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2633
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2634 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2635 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2636 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2637 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2638
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2639 *p = '\0';
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2640 if (PyBytes_Resize(repr, p - PyBytes_AS_STRING(repr))) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2641 Py_DECREF(repr);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2642 return NULL;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2643 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2644 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2645 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2646
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2647 PyObject *PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2648 {
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2649 PyObject *s, *result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2650 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2651 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2652 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2653 }
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2654 s = PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2655 PyUnicode_GET_SIZE(unicode));
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2656
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2657 if (!s)
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2658 return NULL;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2659 result = PyString_FromStringAndSize(PyBytes_AS_STRING(s),
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2660 PyBytes_GET_SIZE(s));
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2661 Py_DECREF(s);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
2662 return result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2663 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2664
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2665 /* --- Raw Unicode Escape Codec ------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2666
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2667 PyObject *PyUnicode_DecodeRawUnicodeEscape(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2668 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2669 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2670 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2671 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2672 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2673 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2674 Py_ssize_t outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2675 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2676 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2677 const char *end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2678 const char *bs;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2679 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2680 PyObject *exc = NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2681
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2682 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2683 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2684 length after conversion to the true value. (But decoding error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2685 handler might have to resize the string) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2686 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2687 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2688 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2689 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2690 return (PyObject *)v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2691 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2692 end = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2693 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2694 unsigned char c;
22423
b523a470a0ce Do not insert characters for unicode-escape decoders if the error mode
Martin v. Löwis <martin@v.loewis.de>
parents: 22087
diff changeset
2695 Py_UCS4 x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2696 int i;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2697 int count;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2698
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2699 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2700 if (*s != '\\') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2701 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2702 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2703 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2704 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2705
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2706 /* \u-escapes are only interpreted iff the number of leading
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2707 backslashes if odd */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2708 bs = s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2709 for (;s < end;) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2710 if (*s != '\\')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2711 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2712 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2713 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2714 if (((s - bs) & 1) == 0 ||
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2715 s >= end ||
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2716 (*s != 'u' && *s != 'U')) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2717 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2718 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2719 p--;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2720 count = *s=='u' ? 4 : 8;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2721 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2722
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2723 /* \uXXXX with 4 hex digits, \Uxxxxxxxx with 8 */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2724 outpos = p-PyUnicode_AS_UNICODE(v);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2725 for (x = 0, i = 0; i < count; ++i, ++s) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2726 c = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2727 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2728 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2729 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2730 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2731 "rawunicodeescape", "truncated \\uXXXX",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2732 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2733 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2734 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2735 goto nextByte;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2736 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2737 x = (x<<4) & ~0xF;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2738 if (c >= '0' && c <= '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2739 x += c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2740 else if (c >= 'a' && c <= 'f')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2741 x += 10 + c - 'a';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2742 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2743 x += 10 + c - 'A';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2744 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2745 #ifndef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2746 if (x > 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2747 if (unicode_decode_call_errorhandler(
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2748 errors, &errorHandler,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2749 "rawunicodeescape", "\\Uxxxxxxxx out of range",
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2750 starts, size, &startinpos, &endinpos, &exc, &s,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2751 (PyObject **)&v, &outpos, &p))
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2752 goto onError;
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2753 }
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2754 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2755 *p++ = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2756 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2757 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2758 }
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2759 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2760 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2761 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2762 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2763 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2764
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2765 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2766 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2767 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2768 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2769 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2770 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2771
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2772 PyObject *PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2773 Py_ssize_t size)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2774 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2775 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2776 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2777 char *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2778
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2779 #ifdef Py_UNICODE_WIDE
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2780 repr = PyBytes_FromStringAndSize(NULL, 10 * size);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2781 #else
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2782 repr = PyBytes_FromStringAndSize(NULL, 6 * size);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2783 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2784 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2785 return NULL;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2786 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2787 return repr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2788
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2789 p = q = PyBytes_AS_STRING(repr);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2790 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2791 Py_UNICODE ch = *s++;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2792 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2793 /* Map 32-bit characters to '\Uxxxxxxxx' */
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2794 if (ch >= 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2795 *p++ = '\\';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2796 *p++ = 'U';
41704
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2797 *p++ = hexdigits[(ch >> 28) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2798 *p++ = hexdigits[(ch >> 24) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2799 *p++ = hexdigits[(ch >> 20) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2800 *p++ = hexdigits[(ch >> 16) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2801 *p++ = hexdigits[(ch >> 12) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2802 *p++ = hexdigits[(ch >> 8) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2803 *p++ = hexdigits[(ch >> 4) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2804 *p++ = hexdigits[ch & 15];
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2805 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2806 else
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2807 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2808 /* Map 16-bit characters to '\uxxxx' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2809 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2810 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2811 *p++ = 'u';
41704
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2812 *p++ = hexdigits[(ch >> 12) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2813 *p++ = hexdigits[(ch >> 8) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2814 *p++ = hexdigits[(ch >> 4) & 0xf];
d237f47f293f Reuse static global hexdigits array.
Walter Dörwald <walter@livinglogic.de>
parents: 41703
diff changeset
2815 *p++ = hexdigits[ch & 15];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2816 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2817 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2818 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2819 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2820 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2821 *p = '\0';
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2822 if (PyBytes_Resize(repr, p - q)) {
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2823 Py_DECREF(repr);
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2824 return NULL;
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2825 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2826 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2827 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2828
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2829 PyObject *PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2830 {
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2831 PyObject *s, *result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2832 if (!PyUnicode_Check(unicode)) {
41705
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2833 PyErr_BadArgument();
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2834 return NULL;
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2835 }
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2836 s = PyUnicode_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2837 PyUnicode_GET_SIZE(unicode));
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2838
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2839 if (!s)
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2840 return NULL;
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2841 result = PyString_FromStringAndSize(PyBytes_AS_STRING(s),
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2842 PyBytes_GET_SIZE(s));
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2843 Py_DECREF(s);
917040dfec58 Change PyUnicode_EncodeRawUnicodeEscape() to return bytes
Walter Dörwald <walter@livinglogic.de>
parents: 41704
diff changeset
2844 return result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2845 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2846
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2847 /* --- Unicode Internal Codec ------------------------------------------- */
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2848
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2849 PyObject *_PyUnicode_DecodeUnicodeInternal(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2850 Py_ssize_t size,
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2851 const char *errors)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2852 {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2853 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2854 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2855 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2856 Py_ssize_t outpos;
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2857 PyUnicodeObject *v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2858 Py_UNICODE *p;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2859 const char *end;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2860 const char *reason;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2861 PyObject *errorHandler = NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2862 PyObject *exc = NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2863
36236
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2864 #ifdef Py_UNICODE_WIDE
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2865 Py_UNICODE unimax = PyUnicode_GetMax();
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2866 #endif
ecd423718a32 Fix icc warnings: remove (sometimes) unused variable conditionally
Neal Norwitz <nnorwitz@gmail.com>
parents: 36203
diff changeset
2867
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
2868 /* XXX overflow detection missing */
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2869 v = _PyUnicode_New((size+Py_UNICODE_SIZE-1)/ Py_UNICODE_SIZE);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2870 if (v == NULL)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2871 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2872 if (PyUnicode_GetSize((PyObject *)v) == 0)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2873 return (PyObject *)v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2874 p = PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2875 end = s + size;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2876
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2877 while (s < end) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
2878 memcpy(p, s, sizeof(Py_UNICODE));
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2879 /* We have to sanity check the raw data, otherwise doom looms for
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2880 some malformed UCS-4 data. */
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2881 if (
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2882 #ifdef Py_UNICODE_WIDE
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2883 *p > unimax || *p < 0 ||
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2884 #endif
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2885 end-s < Py_UNICODE_SIZE
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2886 )
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2887 {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2888 startinpos = s - starts;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2889 if (end-s < Py_UNICODE_SIZE) {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2890 endinpos = end-starts;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2891 reason = "truncated input";
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2892 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2893 else {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2894 endinpos = s - starts + Py_UNICODE_SIZE;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2895 reason = "illegal code point (> 0x10FFFF)";
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2896 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2897 outpos = p - PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2898 if (unicode_decode_call_errorhandler(
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2899 errors, &errorHandler,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2900 "unicode_internal", reason,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2901 starts, size, &startinpos, &endinpos, &exc, &s,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2902 (PyObject **)&v, &outpos, &p)) {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2903 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2904 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2905 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2906 else {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2907 p++;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2908 s += Py_UNICODE_SIZE;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2909 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2910 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2911
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
2912 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2913 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2914 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2915 Py_XDECREF(exc);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2916 return (PyObject *)v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2917
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2918 onError:
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2919 Py_XDECREF(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2920 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2921 Py_XDECREF(exc);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2922 return NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2923 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2924
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2925 /* --- Latin-1 Codec ------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2926
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2927 PyObject *PyUnicode_DecodeLatin1(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2928 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2929 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2930 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2931 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2932 Py_UNICODE *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2933
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2934 /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2935 if (size == 1) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2936 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2937 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2938 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2939
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2940 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2941 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2942 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2943 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2944 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2945 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2946 while (size-- > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2947 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2948 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2949
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2950 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2951 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2952 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2953 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2954
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2955 /* create or adjust a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2956 static void make_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2957 const char *encoding,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2958 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2959 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2960 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2961 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2962 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2963 *exceptionObject = PyUnicodeEncodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2964 encoding, unicode, size, startpos, endpos, reason);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2965 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2966 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2967 if (PyUnicodeEncodeError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2968 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2969 if (PyUnicodeEncodeError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2970 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2971 if (PyUnicodeEncodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2972 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2973 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2974 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2975 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2976 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2977 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2978 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2979
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2980 /* raises a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2981 static void raise_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2982 const char *encoding,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2983 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
2984 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2985 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2986 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2987 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2988 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2989 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2990 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2991 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2992
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2993 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2994 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2995 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2996 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2997 static PyObject *unicode_encode_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2998 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2999 const char *encoding, const char *reason,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3000 const Py_UNICODE *unicode, Py_ssize_t size, PyObject **exceptionObject,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3001 Py_ssize_t startpos, Py_ssize_t endpos,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3002 Py_ssize_t *newpos)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3003 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3004 static char *argparse = "O!n;encoding error handler must return (unicode, int) tuple";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3005
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3006 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3007 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3008
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3009 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3010 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3011 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3012 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3013 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3014
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3015 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3016 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3017 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3018 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3019
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3020 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3021 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3022 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3023 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3024 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3025 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3026 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3027 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3028 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3029 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3030 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3031 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3032 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3033 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3034 if (*newpos<0)
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3035 *newpos = size+*newpos;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3036 if (*newpos<0 || *newpos>size) {
36510
0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat.
Martin v. Löwis <martin@v.loewis.de>
parents: 36508
diff changeset
3037 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", *newpos);
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3038 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3039 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3040 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3041 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3042 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3043 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3044 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3045
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3046 static PyObject *unicode_encode_ucs1(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3047 Py_ssize_t size,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3048 const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3049 int limit)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3050 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3051 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3052 PyObject *res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3053 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3054 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3055 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3056 /* pointer to the beginning of the unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3057 /* const Py_UNICODE *badp = NULL; */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3058 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3059 char *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3060 /* current output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3061 Py_ssize_t respos = 0;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3062 Py_ssize_t ressize;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3063 const char *encoding = (limit == 256) ? "latin-1" : "ascii";
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3064 const char *reason = (limit == 256) ? "ordinal not in range(256)" : "ordinal not in range(128)";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3065 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3066 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3067 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3068 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3069 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3070
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3071 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3072 replacements, if we need more, we'll resize */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3073 res = PyBytes_FromStringAndSize(NULL, size);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3074 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3075 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3076 if (size == 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3077 return res;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3078 str = PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3079 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3080
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3081 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3082 Py_UNICODE c = *p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3083
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3084 /* can we encode this? */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3085 if (c<limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3086 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3087 *str++ = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3088 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3089 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3090 else {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3091 Py_ssize_t unicodepos = p-startp;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3092 Py_ssize_t requiredsize;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3093 PyObject *repunicode;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3094 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3095 Py_ssize_t newpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3096 Py_ssize_t respos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3097 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3098 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3099 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3100 const Py_UNICODE *collend = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3101 /* find all unecodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3102 while ((collend < endp) && ((*collend)>=limit))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3103 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3104 /* cache callback name lookup (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3105 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3106 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3107 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3108 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3109 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3110 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3111 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3112 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3113 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3114 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3115 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3116 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3117 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3118 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3119 raise_encode_exception(&exc, encoding, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3120 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3121 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3122 while (collstart++<collend)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3123 *str++ = '?'; /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3124 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3125 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3126 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3127 case 4: /* xmlcharrefreplace */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3128 respos = str - PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3129 /* determine replacement size (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3130 for (p = collstart, repsize = 0; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3131 if (*p<10)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3132 repsize += 2+1+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3133 else if (*p<100)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3134 repsize += 2+2+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3135 else if (*p<1000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3136 repsize += 2+3+1;
31248
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
3137 else if (*p<10000)
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
3138 repsize += 2+4+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3139 #ifndef Py_UNICODE_WIDE
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3140 else
31248
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
3141 repsize += 2+5+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3142 #else
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3143 else if (*p<100000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3144 repsize += 2+5+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3145 else if (*p<1000000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3146 repsize += 2+6+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3147 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3148 repsize += 2+7+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
3149 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3150 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3151 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3152 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3153 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3154 requiredsize = 2*ressize;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3155 if (PyBytes_Resize(res, requiredsize))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3156 goto onError;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3157 str = PyBytes_AS_STRING(res) + respos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3158 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3159 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3160 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3161 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3162 str += sprintf(str, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3163 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3164 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3165 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3166 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3167 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3168 encoding, reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3169 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3170 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3171 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3172 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3173 have+the replacement+the rest of the string, so
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3174 we won't have to check space for encodable characters) */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3175 respos = str - PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3176 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3177 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3178 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3179 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3180 requiredsize = 2*ressize;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3181 if (PyBytes_Resize(res, requiredsize)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3182 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3183 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3184 }
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3185 str = PyBytes_AS_STRING(res) + respos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3186 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3187 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3188 /* check if there is anything unencodable in the replacement
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3189 and copy it to the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3190 for (uni2 = PyUnicode_AS_UNICODE(repunicode);repsize-->0; ++uni2, ++str) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3191 c = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3192 if (c >= limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3193 raise_encode_exception(&exc, encoding, startp, size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3194 unicodepos, unicodepos+1, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3195 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3196 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3197 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3198 *str = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3199 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3200 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3201 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3202 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3203 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3204 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3205 /* Resize if we allocated to much */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3206 respos = str - PyBytes_AS_STRING(res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3207 if (respos<ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3208 /* If this falls res will be NULL */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3209 PyBytes_Resize(res, respos);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3210 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3211 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3212 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3213
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3214 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3215 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3216 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3217 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3218 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3219 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3220
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3221 PyObject *PyUnicode_EncodeLatin1(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3222 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3223 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3224 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3225 return unicode_encode_ucs1(p, size, errors, 256);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3226 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3227
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3228 PyObject *PyUnicode_AsLatin1String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3229 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3230 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3231 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3232 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3233 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3234 return PyUnicode_EncodeLatin1(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3235 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3236 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3237 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3239 /* --- 7-bit ASCII Codec -------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3240
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3241 PyObject *PyUnicode_DecodeASCII(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3242 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3243 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3244 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3245 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3246 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3247 Py_UNICODE *p;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3248 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3249 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3250 Py_ssize_t outpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3251 const char *e;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3252 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3253 PyObject *exc = NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3254
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3255 /* ASCII is equivalent to the first 128 ordinals in Unicode. */
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3256 if (size == 1 && *(unsigned char*)s < 128) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3257 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3258 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3259 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3260
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3261 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3262 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3263 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3264 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3265 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3266 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3267 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3268 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3269 register unsigned char c = (unsigned char)*s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3270 if (c < 128) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3271 *p++ = c;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3272 ++s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3273 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3274 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3275 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3276 endinpos = startinpos + 1;
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
3277 outpos = p - (Py_UNICODE *)PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3278 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3279 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3280 "ascii", "ordinal not in range(128)",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3281 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3282 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3283 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3284 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3285 }
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
3286 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3287 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
3288 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3289 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3290 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3291 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3292
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3293 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3294 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3295 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3296 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3297 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3298 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3299
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3300 PyObject *PyUnicode_EncodeASCII(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3301 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3302 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3303 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3304 return unicode_encode_ucs1(p, size, errors, 128);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3305 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3306
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3307 PyObject *PyUnicode_AsASCIIString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3308 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3309 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3310 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3311 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3312 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3313 return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3314 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3315 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3316 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3317
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
3318 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
3319
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3320 /* --- MBCS codecs for Windows -------------------------------------------- */
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
3321
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3322 #if SIZEOF_INT < SIZEOF_SSIZE_T
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3323 #define NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3324 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3325
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3326 /* XXX This code is limited to "true" double-byte encodings, as
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3327 a) it assumes an incomplete character consists of a single byte, and
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3328 b) IsDBCSLeadByte (probably) does not work for non-DBCS multi-byte
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3329 encodings, see IsDBCSLeadByteEx documentation. */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3330
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3331 static int is_dbcs_lead_byte(const char *s, int offset)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3332 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3333 const char *curr = s + offset;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3334
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3335 if (IsDBCSLeadByte(*curr)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3336 const char *prev = CharPrev(s, curr);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3337 return (prev == curr) || !IsDBCSLeadByte(*prev) || (curr - prev == 2);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3338 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3339 return 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3340 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3341
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3342 /*
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3343 * Decode MBCS string into unicode object. If 'final' is set, converts
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3344 * trailing lead-byte too. Returns consumed size if succeed, -1 otherwise.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3345 */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3346 static int decode_mbcs(PyUnicodeObject **v,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3347 const char *s, /* MBCS string */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3348 int size, /* sizeof MBCS string */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3349 int final)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3350 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3351 Py_UNICODE *p;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3352 Py_ssize_t n = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3353 int usize = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3354
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3355 assert(size >= 0);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3356
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3357 /* Skip trailing lead-byte unless 'final' is set */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3358 if (!final && size >= 1 && is_dbcs_lead_byte(s, size - 1))
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3359 --size;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3360
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3361 /* First get the size of the result */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3362 if (size > 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3363 usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3364 if (usize == 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3365 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3366 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3367 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3368 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3369
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3370 if (*v == NULL) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3371 /* Create unicode object */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3372 *v = _PyUnicode_New(usize);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3373 if (*v == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3374 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3375 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3376 else {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3377 /* Extend unicode object */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3378 n = PyUnicode_GET_SIZE(*v);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3379 if (_PyUnicode_Resize(v, n + usize) < 0)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3380 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3381 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3382
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3383 /* Do the conversion */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3384 if (size > 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3385 p = PyUnicode_AS_UNICODE(*v) + n;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3386 if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3387 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3388 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3389 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3390 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3391
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3392 return size;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3393 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3394
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3395 PyObject *PyUnicode_DecodeMBCSStateful(const char *s,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3396 Py_ssize_t size,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3397 const char *errors,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3398 Py_ssize_t *consumed)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3399 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3400 PyUnicodeObject *v = NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3401 int done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3402
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3403 if (consumed)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3404 *consumed = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3405
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3406 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3407 retry:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3408 if (size > INT_MAX)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3409 done = decode_mbcs(&v, s, INT_MAX, 0);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3410 else
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3411 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3412 done = decode_mbcs(&v, s, (int)size, !consumed);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3413
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3414 if (done < 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3415 Py_XDECREF(v);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3416 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3417 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3418
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3419 if (consumed)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3420 *consumed += done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3421
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3422 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3423 if (size > INT_MAX) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3424 s += done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3425 size -= done;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3426 goto retry;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3427 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3428 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3429
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3430 return (PyObject *)v;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3431 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3432
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3433 PyObject *PyUnicode_DecodeMBCS(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3434 Py_ssize_t size,
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3435 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3436 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3437 return PyUnicode_DecodeMBCSStateful(s, size, errors, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3438 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3439
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3440 /*
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3441 * Convert unicode into string object (MBCS).
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3442 * Returns 0 if succeed, -1 otherwise.
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3443 */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3444 static int encode_mbcs(PyObject **repr,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3445 const Py_UNICODE *p, /* unicode */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3446 int size) /* size of unicode */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3447 {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3448 int mbcssize = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3449 Py_ssize_t n = 0;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3450
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3451 assert(size >= 0);
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3452
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3453 /* First get the size of the result */
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3454 if (size > 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3455 mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3456 if (mbcssize == 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3457 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3458 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3459 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3460 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3461
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3462 if (*repr == NULL) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3463 /* Create string object */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3464 *repr = PyBytes_FromStringAndSize(NULL, mbcssize);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3465 if (*repr == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3466 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3467 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3468 else {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3469 /* Extend string object */
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3470 n = PyBytes_Size(*repr);
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3471 if (PyBytes_Resize(*repr, n + mbcssize) < 0)
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3472 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3473 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3474
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3475 /* Do the conversion */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3476 if (size > 0) {
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
3477 char *s = PyBytes_AS_STRING(*repr) + n;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3478 if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3479 PyErr_SetFromWindowsErrWithFilename(0, NULL);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3480 return -1;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3481 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3482 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3483
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3484 return 0;
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3485 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3486
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3487 PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3488 Py_ssize_t size,
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3489 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3490 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3491 PyObject *repr = NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3492 int ret;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3493
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3494 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3495 retry:
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3496 if (size > INT_MAX)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3497 ret = encode_mbcs(&repr, p, INT_MAX);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3498 else
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3499 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3500 ret = encode_mbcs(&repr, p, (int)size);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3501
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3502 if (ret < 0) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3503 Py_XDECREF(repr);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3504 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3505 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3506
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3507 #ifdef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3508 if (size > INT_MAX) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3509 p += INT_MAX;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3510 size -= INT_MAX;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3511 goto retry;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3512 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3513 #endif
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3514
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3515 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3516 }
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
3517
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3518 PyObject *PyUnicode_AsMBCSString(PyObject *unicode)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3519 {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3520 if (!PyUnicode_Check(unicode)) {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3521 PyErr_BadArgument();
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3522 return NULL;
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3523 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3524 return PyUnicode_EncodeMBCS(PyUnicode_AS_UNICODE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3525 PyUnicode_GET_SIZE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3526 NULL);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3527 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
3528
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3529 #undef NEED_RETRY
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
3530
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
3531 #endif /* MS_WINDOWS */
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
3532
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3533 /* --- Character Mapping Codec -------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3534
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3535 PyObject *PyUnicode_DecodeCharmap(const char *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3536 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3537 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3538 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3539 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3540 const char *starts = s;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3541 Py_ssize_t startinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3542 Py_ssize_t endinpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3543 Py_ssize_t outpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3544 const char *e;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3545 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3546 Py_UNICODE *p;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3547 Py_ssize_t extrachars = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3548 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3549 PyObject *exc = NULL;
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3550 Py_UNICODE *mapstring = NULL;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3551 Py_ssize_t maplen = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3552
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3553 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3554 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3555 return PyUnicode_DecodeLatin1(s, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3556
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3557 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3558 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3559 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3560 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3561 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3562 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3563 e = s + size;
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3564 if (PyUnicode_CheckExact(mapping)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3565 mapstring = PyUnicode_AS_UNICODE(mapping);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3566 maplen = PyUnicode_GET_SIZE(mapping);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3567 while (s < e) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3568 unsigned char ch = *s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3569 Py_UNICODE x = 0xfffe; /* illegal value */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3570
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3571 if (ch < maplen)
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3572 x = mapstring[ch];
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3573
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3574 if (x == 0xfffe) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3575 /* undefined mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3576 outpos = p-PyUnicode_AS_UNICODE(v);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3577 startinpos = s-starts;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3578 endinpos = startinpos+1;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3579 if (unicode_decode_call_errorhandler(
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3580 errors, &errorHandler,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3581 "charmap", "character maps to <undefined>",
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3582 starts, size, &startinpos, &endinpos, &exc, &s,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3583 (PyObject **)&v, &outpos, &p)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3584 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3585 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3586 continue;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3587 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3588 *p++ = x;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3589 ++s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3590 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3591 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3592 else {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3593 while (s < e) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3594 unsigned char ch = *s;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3595 PyObject *w, *x;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3596
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3597 /* Get mapping (char ordinal -> integer, Unicode char or None) */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3598 w = PyInt_FromLong((long)ch);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3599 if (w == NULL)
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3600 goto onError;
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3601 x = PyObject_GetItem(mapping, w);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3602 Py_DECREF(w);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3603 if (x == NULL) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3604 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3605 /* No mapping found means: mapping is undefined. */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3606 PyErr_Clear();
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3607 x = Py_None;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3608 Py_INCREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3609 } else
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3610 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3611 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3612
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3613 /* Apply mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3614 if (PyInt_Check(x)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3615 long value = PyInt_AS_LONG(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3616 if (value < 0 || value > 65535) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3617 PyErr_SetString(PyExc_TypeError,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3618 "character mapping must be in range(65536)");
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3619 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3620 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3621 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3622 *p++ = (Py_UNICODE)value;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3623 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3624 else if (x == Py_None) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3625 /* undefined mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3626 outpos = p-PyUnicode_AS_UNICODE(v);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3627 startinpos = s-starts;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3628 endinpos = startinpos+1;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3629 if (unicode_decode_call_errorhandler(
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3630 errors, &errorHandler,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3631 "charmap", "character maps to <undefined>",
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3632 starts, size, &startinpos, &endinpos, &exc, &s,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3633 (PyObject **)&v, &outpos, &p)) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3634 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3635 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3636 }
35962
73ebdc62bf8f Fix leaked reference to None.
Walter Dörwald <walter@livinglogic.de>
parents: 35827
diff changeset
3637 Py_DECREF(x);
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3638 continue;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3639 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3640 else if (PyUnicode_Check(x)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3641 Py_ssize_t targetsize = PyUnicode_GET_SIZE(x);
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3642
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3643 if (targetsize == 1)
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3644 /* 1-1 mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3645 *p++ = *PyUnicode_AS_UNICODE(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3646
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3647 else if (targetsize > 1) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3648 /* 1-n mapping */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3649 if (targetsize > extrachars) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3650 /* resize first */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3651 Py_ssize_t oldpos = p - PyUnicode_AS_UNICODE(v);
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
3652 Py_ssize_t needed = (targetsize - extrachars) + \
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3653 (targetsize << 2);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3654 extrachars += needed;
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
3655 /* XXX overflow detection missing */
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3656 if (_PyUnicode_Resize(&v,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3657 PyUnicode_GET_SIZE(v) + needed) < 0) {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3658 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3659 goto onError;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3660 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3661 p = PyUnicode_AS_UNICODE(v) + oldpos;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3662 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3663 Py_UNICODE_COPY(p,
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3664 PyUnicode_AS_UNICODE(x),
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3665 targetsize);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3666 p += targetsize;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3667 extrachars -= targetsize;
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3668 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3669 /* 1-0 mapping: skip the character */
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3670 }
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3671 else {
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3672 /* wrong return value */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3673 PyErr_SetString(PyExc_TypeError,
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3674 "character mapping must return integer, None or unicode");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3675 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3676 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3677 }
35702
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3678 Py_DECREF(x);
22a720b35314 Part of SF patch #1313939: Speedup charmap decoding by extending
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
3679 ++s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3680 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3681 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3682 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
3683 if (_PyUnicode_Resize(&v, p - PyUnicode_AS_UNICODE(v)) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3684 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3685 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3686 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3687 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3688
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3689 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3690 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3691 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3692 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3693 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3694 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3695
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3696 /* Charmap encoding: the lookup table */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3697
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3698 struct encoding_map{
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3699 PyObject_HEAD
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3700 unsigned char level1[32];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3701 int count2, count3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3702 unsigned char level23[1];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3703 };
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3704
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3705 static PyObject*
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3706 encoding_map_size(PyObject *obj, PyObject* args)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3707 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3708 struct encoding_map *map = (struct encoding_map*)obj;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3709 return PyInt_FromLong(sizeof(*map) - 1 + 16*map->count2 +
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3710 128*map->count3);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3711 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3712
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3713 static PyMethodDef encoding_map_methods[] = {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3714 {"size", encoding_map_size, METH_NOARGS,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3715 PyDoc_STR("Return the size (in bytes) of this object") },
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3716 { 0 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3717 };
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3718
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3719 static void
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3720 encoding_map_dealloc(PyObject* o)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3721 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3722 PyObject_FREE(o);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3723 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3724
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3725 static PyTypeObject EncodingMapType = {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3726 PyObject_HEAD_INIT(NULL)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3727 0, /*ob_size*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3728 "EncodingMap", /*tp_name*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3729 sizeof(struct encoding_map), /*tp_basicsize*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3730 0, /*tp_itemsize*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3731 /* methods */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3732 encoding_map_dealloc, /*tp_dealloc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3733 0, /*tp_print*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3734 0, /*tp_getattr*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3735 0, /*tp_setattr*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3736 0, /*tp_compare*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3737 0, /*tp_repr*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3738 0, /*tp_as_number*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3739 0, /*tp_as_sequence*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3740 0, /*tp_as_mapping*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3741 0, /*tp_hash*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3742 0, /*tp_call*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3743 0, /*tp_str*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3744 0, /*tp_getattro*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3745 0, /*tp_setattro*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3746 0, /*tp_as_buffer*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3747 Py_TPFLAGS_DEFAULT, /*tp_flags*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3748 0, /*tp_doc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3749 0, /*tp_traverse*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3750 0, /*tp_clear*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3751 0, /*tp_richcompare*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3752 0, /*tp_weaklistoffset*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3753 0, /*tp_iter*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3754 0, /*tp_iternext*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3755 encoding_map_methods, /*tp_methods*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3756 0, /*tp_members*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3757 0, /*tp_getset*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3758 0, /*tp_base*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3759 0, /*tp_dict*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3760 0, /*tp_descr_get*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3761 0, /*tp_descr_set*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3762 0, /*tp_dictoffset*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3763 0, /*tp_init*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3764 0, /*tp_alloc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3765 0, /*tp_new*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3766 0, /*tp_free*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3767 0, /*tp_is_gc*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3768 };
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3769
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3770 PyObject*
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3771 PyUnicode_BuildEncodingMap(PyObject* string)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3772 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3773 Py_UNICODE *decode;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3774 PyObject *result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3775 struct encoding_map *mresult;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3776 int i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3777 int need_dict = 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3778 unsigned char level1[32];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3779 unsigned char level2[512];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3780 unsigned char *mlevel1, *mlevel2, *mlevel3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3781 int count2 = 0, count3 = 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3782
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3783 if (!PyUnicode_Check(string) || PyUnicode_GetSize(string) != 256) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3784 PyErr_BadArgument();
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3785 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3786 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3787 decode = PyUnicode_AS_UNICODE(string);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3788 memset(level1, 0xFF, sizeof level1);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3789 memset(level2, 0xFF, sizeof level2);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3790
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3791 /* If there isn't a one-to-one mapping of NULL to \0,
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3792 or if there are non-BMP characters, we need to use
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3793 a mapping dictionary. */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3794 if (decode[0] != 0)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3795 need_dict = 1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3796 for (i = 1; i < 256; i++) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3797 int l1, l2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3798 if (decode[i] == 0
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3799 #ifdef Py_UNICODE_WIDE
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3800 || decode[i] > 0xFFFF
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3801 #endif
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3802 ) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3803 need_dict = 1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3804 break;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3805 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3806 if (decode[i] == 0xFFFE)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3807 /* unmapped character */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3808 continue;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3809 l1 = decode[i] >> 11;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3810 l2 = decode[i] >> 7;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3811 if (level1[l1] == 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3812 level1[l1] = count2++;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3813 if (level2[l2] == 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3814 level2[l2] = count3++;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3815 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3816
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3817 if (count2 >= 0xFF || count3 >= 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3818 need_dict = 1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3819
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3820 if (need_dict) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3821 PyObject *result = PyDict_New();
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3822 PyObject *key, *value;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3823 if (!result)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3824 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3825 for (i = 0; i < 256; i++) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3826 key = value = NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3827 key = PyInt_FromLong(decode[i]);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3828 value = PyInt_FromLong(i);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3829 if (!key || !value)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3830 goto failed1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3831 if (PyDict_SetItem(result, key, value) == -1)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3832 goto failed1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3833 Py_DECREF(key);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3834 Py_DECREF(value);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3835 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3836 return result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3837 failed1:
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3838 Py_XDECREF(key);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3839 Py_XDECREF(value);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3840 Py_DECREF(result);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3841 return NULL;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3842 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3843
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3844 /* Create a three-level trie */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3845 result = PyObject_MALLOC(sizeof(struct encoding_map) +
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3846 16*count2 + 128*count3 - 1);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3847 if (!result)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3848 return PyErr_NoMemory();
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3849 PyObject_Init(result, &EncodingMapType);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3850 mresult = (struct encoding_map*)result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3851 mresult->count2 = count2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3852 mresult->count3 = count3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3853 mlevel1 = mresult->level1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3854 mlevel2 = mresult->level23;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3855 mlevel3 = mresult->level23 + 16*count2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3856 memcpy(mlevel1, level1, 32);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3857 memset(mlevel2, 0xFF, 16*count2);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3858 memset(mlevel3, 0, 128*count3);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3859 count3 = 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3860 for (i = 1; i < 256; i++) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3861 int o1, o2, o3, i2, i3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3862 if (decode[i] == 0xFFFE)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3863 /* unmapped character */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3864 continue;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3865 o1 = decode[i]>>11;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3866 o2 = (decode[i]>>7) & 0xF;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3867 i2 = 16*mlevel1[o1] + o2;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3868 if (mlevel2[i2] == 0xFF)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3869 mlevel2[i2] = count3++;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3870 o3 = decode[i] & 0x7F;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3871 i3 = 128*mlevel2[i2] + o3;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3872 mlevel3[i3] = i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3873 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3874 return result;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3875 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3876
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3877 static int
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3878 encoding_map_lookup(Py_UNICODE c, PyObject *mapping)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3879 {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3880 struct encoding_map *map = (struct encoding_map*)mapping;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3881 int l1 = c>>11;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3882 int l2 = (c>>7) & 0xF;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3883 int l3 = c & 0x7F;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3884 int i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3885
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3886 #ifdef Py_UNICODE_WIDE
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3887 if (c > 0xFFFF) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3888 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3889 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3890 #endif
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3891 if (c == 0)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3892 return 0;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3893 /* level 1*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3894 i = map->level1[l1];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3895 if (i == 0xFF) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3896 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3897 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3898 /* level 2*/
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3899 i = map->level23[16*i+l2];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3900 if (i == 0xFF) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3901 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3902 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3903 /* level 3 */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3904 i = map->level23[16*map->count2 + 128*i + l3];
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3905 if (i == 0) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3906 return -1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3907 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3908 return i;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3909 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3910
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3911 /* Lookup the character ch in the mapping. If the character
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3912 can't be found, Py_None is returned (or NULL, if another
35806
44fe157255bc fix typos, mostly in comments
Fred Drake <fdrake@acm.org>
parents: 35758
diff changeset
3913 error occurred). */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3914 static PyObject *charmapencode_lookup(Py_UNICODE c, PyObject *mapping)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3915 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3916 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3917 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3918
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3919 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3920 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3921 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3922 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3923 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3924 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3925 /* No mapping found means: mapping is undefined. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3926 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3927 x = Py_None;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3928 Py_INCREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3929 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3930 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3931 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3932 }
27175
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping
Walter Dörwald <walter@livinglogic.de>
parents: 27170
diff changeset
3933 else if (x == Py_None)
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping
Walter Dörwald <walter@livinglogic.de>
parents: 27170
diff changeset
3934 return x;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3935 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3936 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3937 if (value < 0 || value > 255) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3938 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3939 "character mapping must be in range(256)");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3940 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3941 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3942 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3943 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3944 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3945 else if (PyString_Check(x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3946 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3947 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3948 /* wrong return value */
41658
7fe6afcbd3f2 Mention type in the exception message.
Walter Dörwald <walter@livinglogic.de>
parents: 41629
diff changeset
3949 PyErr_Format(PyExc_TypeError,
7fe6afcbd3f2 Mention type in the exception message.
Walter Dörwald <walter@livinglogic.de>
parents: 41629
diff changeset
3950 "character mapping must return integer, None or str8, not %.400s",
7fe6afcbd3f2 Mention type in the exception message.
Walter Dörwald <walter@livinglogic.de>
parents: 41629
diff changeset
3951 x->ob_type->tp_name);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3952 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3953 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3954 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3955 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3956
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3957 static int
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3958 charmapencode_resize(PyObject *outobj, Py_ssize_t *outpos, Py_ssize_t requiredsize)
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3959 {
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3960 Py_ssize_t outsize = PyBytes_GET_SIZE( outobj);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3961 /* exponentially overallocate to minimize reallocations */
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3962 if (requiredsize < 2*outsize)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3963 requiredsize = 2*outsize;
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3964 if (PyBytes_Resize(outobj, requiredsize)) {
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3965 Py_DECREF(outobj);
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3966 return -1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3967 }
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3968 return 0;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3969 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3970
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3971 typedef enum charmapencode_result {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3972 enc_SUCCESS, enc_FAILED, enc_EXCEPTION
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3973 }charmapencode_result;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3974 /* lookup the character, put the result in the output string and adjust
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3975 various state variables. Resize the output bytes object if not enough
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3976 space is available. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3977 was put in the output buffer, or Py_None, if the mapping was undefined
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3978 (in which case no character was written) or NULL, if a
35827
ab3d1e6a0237 Another comment typo fix
Andrew M. Kuchling <amk@amk.ca>
parents: 35826
diff changeset
3979 reallocation error occurred. The caller must decref the result */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3980 static
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3981 charmapencode_result charmapencode_output(Py_UNICODE c, PyObject *mapping,
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3982 PyObject *outobj, Py_ssize_t *outpos)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3983 {
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3984 PyObject *rep;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3985 char *outstart;
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3986 Py_ssize_t outsize = PyBytes_GET_SIZE(outobj);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3987
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3988 if (mapping->ob_type == &EncodingMapType) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3989 int res = encoding_map_lookup(c, mapping);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3990 Py_ssize_t requiredsize = *outpos+1;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3991 if (res == -1)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3992 return enc_FAILED;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3993 if (outsize<requiredsize)
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3994 if (charmapencode_resize(outobj, outpos, requiredsize))
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3995 return enc_EXCEPTION;
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
3996 outstart = PyBytes_AS_STRING(outobj);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3997 outstart[(*outpos)++] = (char)res;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3998 return enc_SUCCESS;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
3999 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4000
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4001 rep = charmapencode_lookup(c, mapping);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4002 if (rep==NULL)
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4003 return enc_EXCEPTION;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4004 else if (rep==Py_None) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4005 Py_DECREF(rep);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4006 return enc_FAILED;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4007 } else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4008 if (PyInt_Check(rep)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4009 Py_ssize_t requiredsize = *outpos+1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4010 if (outsize<requiredsize)
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4011 if (charmapencode_resize(outobj, outpos, requiredsize)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4012 Py_DECREF(rep);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4013 return enc_EXCEPTION;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4014 }
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4015 outstart = PyBytes_AS_STRING(outobj);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4016 outstart[(*outpos)++] = (char)PyInt_AS_LONG(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4017 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4018 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4019 const char *repchars = PyString_AS_STRING(rep);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4020 Py_ssize_t repsize = PyString_GET_SIZE(rep);
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4021 Py_ssize_t requiredsize = *outpos+repsize;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4022 if (outsize<requiredsize)
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4023 if (charmapencode_resize(outobj, outpos, requiredsize)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4024 Py_DECREF(rep);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4025 return enc_EXCEPTION;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4026 }
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4027 outstart = PyBytes_AS_STRING(outobj);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4028 memcpy(outstart + *outpos, repchars, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4029 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4030 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4031 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4032 Py_DECREF(rep);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4033 return enc_SUCCESS;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4034 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4035
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4036 /* handle an error in PyUnicode_EncodeCharmap
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4037 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4038 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4039 int charmap_encoding_error(
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4040 const Py_UNICODE *p, Py_ssize_t size, Py_ssize_t *inpos, PyObject *mapping,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4041 PyObject **exceptionObject,
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
4042 int *known_errorHandler, PyObject **errorHandler, const char *errors,
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4043 PyObject *res, Py_ssize_t *respos)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4044 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4045 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4046 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4047 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4048 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4049 /* startpos for collecting unencodable chars */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4050 Py_ssize_t collstartpos = *inpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4051 Py_ssize_t collendpos = *inpos+1;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4052 Py_ssize_t collpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4053 char *encoding = "charmap";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4054 char *reason = "character maps to <undefined>";
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4055 charmapencode_result x;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4056
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4057 /* find all unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4058 while (collendpos < size) {
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4059 PyObject *rep;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4060 if (mapping->ob_type == &EncodingMapType) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4061 int res = encoding_map_lookup(p[collendpos], mapping);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4062 if (res != -1)
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4063 break;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4064 ++collendpos;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4065 continue;
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4066 }
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4067
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4068 rep = charmapencode_lookup(p[collendpos], mapping);
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4069 if (rep==NULL)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4070 return -1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4071 else if (rep!=Py_None) {
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4072 Py_DECREF(rep);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4073 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4074 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4075 Py_DECREF(rep);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4076 ++collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4077 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4078 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4079 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4080 if (*known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4081 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4082 *known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4083 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4084 *known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4085 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4086 *known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4087 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4088 *known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4089 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4090 *known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4091 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4092 switch (*known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4093 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4094 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4095 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4096 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4097 for (collpos = collstartpos; collpos<collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4098 x = charmapencode_output('?', mapping, res, respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4099 if (x==enc_EXCEPTION) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4100 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4101 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4102 else if (x==enc_FAILED) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4103 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4104 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4105 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4106 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4107 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4108 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4109 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4110 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4111 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4112 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4113 for (collpos = collstartpos; collpos < collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4114 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4115 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4116 sprintf(buffer, "&#%d;", (int)p[collpos]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4117 for (cp = buffer; *cp; ++cp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4118 x = charmapencode_output(*cp, mapping, res, respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4119 if (x==enc_EXCEPTION)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4120 return -1;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4121 else if (x==enc_FAILED) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4122 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4123 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4124 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4125 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4126 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4127 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4128 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4129 default:
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
4130 repunicode = unicode_encode_call_errorhandler(errors, errorHandler,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4131 encoding, reason, p, size, exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4132 collstartpos, collendpos, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4133 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4134 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4135 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4136 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4137 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4138 x = charmapencode_output(*uni2, mapping, res, respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4139 if (x==enc_EXCEPTION) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4140 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4141 }
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4142 else if (x==enc_FAILED) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4143 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4144 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4145 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4146 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4147 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4148 *inpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4149 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4150 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4151 return 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4152 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4153
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4154 PyObject *PyUnicode_EncodeCharmap(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4155 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4156 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4157 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4158 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4159 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4160 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4161 /* current input position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4162 Py_ssize_t inpos = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4163 /* current output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4164 Py_ssize_t respos = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4165 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4166 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4167 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4168 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4169 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4170 int known_errorHandler = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4171
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4172 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4173 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4174 return PyUnicode_EncodeLatin1(p, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4175
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4176 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4177 replacements, if we need more, we'll resize */
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4178 res = PyBytes_FromStringAndSize(NULL, size);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4179 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4180 goto onError;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
4181 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4182 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4183
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4184 while (inpos<size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4185 /* try to encode it */
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4186 charmapencode_result x = charmapencode_output(p[inpos], mapping, res, &respos);
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4187 if (x==enc_EXCEPTION) /* error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4188 goto onError;
38673
a19c20c5e5c7 Merge the rest of the trunk.
Thomas Wouters <thomas@python.org>
parents: 38453
diff changeset
4189 if (x==enc_FAILED) { /* unencodable character */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4190 if (charmap_encoding_error(p, size, &inpos, mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4191 &exc,
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
4192 &known_errorHandler, &errorHandler, errors,
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4193 res, &respos)) {
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
4194 goto onError;
30162
443533d08783 Fix another refcounting leak in PyUnicode_EncodeCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30160
diff changeset
4195 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4196 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4197 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4198 /* done with this character => adjust input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4199 ++inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4200 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4201
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4202 /* Resize if we allocated to much */
41707
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4203 if (respos<PyBytes_GET_SIZE(res)) {
6026b8a26c05 Change PyUnicode_EncodeCharmap() to return bytes objects
Walter Dörwald <walter@livinglogic.de>
parents: 41705
diff changeset
4204 if (PyBytes_Resize(res, respos))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4205 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4206 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4207 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4208 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4209 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4210
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4211 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4212 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4213 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4214 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4215 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4216 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4217
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4218 PyObject *PyUnicode_AsCharmapString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4219 PyObject *mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4220 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4221 if (!PyUnicode_Check(unicode) || mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4222 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4223 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4224 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4225 return PyUnicode_EncodeCharmap(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4226 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4227 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4228 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4229 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4230
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4231 /* create or adjust a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4232 static void make_translate_exception(PyObject **exceptionObject,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4233 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4234 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4235 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4236 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4237 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4238 *exceptionObject = PyUnicodeTranslateError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4239 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4240 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4241 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4242 if (PyUnicodeTranslateError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4243 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4244 if (PyUnicodeTranslateError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4245 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4246 if (PyUnicodeTranslateError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4247 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4248 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4249 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4250 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4251 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4252 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4253 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4254
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4255 /* raises a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4256 static void raise_translate_exception(PyObject **exceptionObject,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4257 const Py_UNICODE *unicode, Py_ssize_t size,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4258 Py_ssize_t startpos, Py_ssize_t endpos,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4259 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4260 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4261 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4262 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4263 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4264 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4265 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4266
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4267 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4268 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4269 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4270 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4271 static PyObject *unicode_translate_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4272 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4273 const char *reason,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4274 const Py_UNICODE *unicode, Py_ssize_t size, PyObject **exceptionObject,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4275 Py_ssize_t startpos, Py_ssize_t endpos,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4276 Py_ssize_t *newpos)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4277 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
4278 static char *argparse = "O!n;translating error handler must return (unicode, int) tuple";
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
4279
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
4280 Py_ssize_t i_newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4281 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4282 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4283
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4284 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4285 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4286 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4287 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4288 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4289
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4290 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4291 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4292 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4293 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4294
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4295 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4296 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4297 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4298 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4299 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4300 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4301 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4302 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4303 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4304 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4305 &resunicode, &i_newpos)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4306 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4307 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4308 }
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4309 if (i_newpos<0)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4310 *newpos = size+i_newpos;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4311 else
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4312 *newpos = i_newpos;
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4313 if (*newpos<0 || *newpos>size) {
36510
0f3894bb2ada Support %zd in PyErr_Format and PyString_FromFormat.
Martin v. Löwis <martin@v.loewis.de>
parents: 36508
diff changeset
4314 PyErr_Format(PyExc_IndexError, "position %zd from error handler out of bounds", *newpos);
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4315 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4316 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
4317 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4318 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4319 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4320 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4321 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4322
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4323 /* Lookup the character ch in the mapping and put the result in result,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4324 which must be decrefed by the caller.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4325 Return 0 on success, -1 on error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4326 static
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4327 int charmaptranslate_lookup(Py_UNICODE c, PyObject *mapping, PyObject **result)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4328 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4329 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4330 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4331
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4332 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4333 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4334 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4335 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4336 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4337 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4338 /* No mapping found means: use 1:1 mapping. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4339 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4340 *result = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4341 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4342 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4343 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4344 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4345 else if (x == Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4346 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4347 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4348 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4349 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4350 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4351 long max = PyUnicode_GetMax();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4352 if (value < 0 || value > max) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4353 PyErr_Format(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4354 "character mapping must be in range(0x%lx)", max+1);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4355 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4356 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4357 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4358 *result = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4359 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4360 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4361 else if (PyUnicode_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4362 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4363 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4364 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4365 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4366 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4367 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4368 "character mapping must return integer, None or unicode");
30164
7cb4ad5932e6 Fix refcounting leak in charmaptranslate_lookup()
Walter Dörwald <walter@livinglogic.de>
parents: 30162
diff changeset
4369 Py_DECREF(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4370 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4371 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4372 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4373 /* ensure that *outobj is at least requiredsize characters long,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4374 if not reallocate and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4375 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4376 static
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4377 int charmaptranslate_makespace(PyObject **outobj, Py_UNICODE **outp,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4378 Py_ssize_t requiredsize)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4379 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4380 Py_ssize_t oldsize = PyUnicode_GET_SIZE(*outobj);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4381 if (requiredsize > oldsize) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4382 /* remember old output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4383 Py_ssize_t outpos = *outp-PyUnicode_AS_UNICODE(*outobj);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4384 /* exponentially overallocate to minimize reallocations */
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4385 if (requiredsize < 2 * oldsize)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4386 requiredsize = 2 * oldsize;
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
4387 if (_PyUnicode_Resize(outobj, requiredsize) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4388 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4389 *outp = PyUnicode_AS_UNICODE(*outobj) + outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4390 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4391 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4392 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4393 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4394 various state variables. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4395 was put in the output buffer in *result, or Py_None, if the mapping was
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4396 undefined (in which case no character was written).
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4397 The called must decref result.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4398 Return 0 on success, -1 on error. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4399 static
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4400 int charmaptranslate_output(const Py_UNICODE *startinp, const Py_UNICODE *curinp,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4401 Py_ssize_t insize, PyObject *mapping, PyObject **outobj, Py_UNICODE **outp,
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4402 PyObject **res)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4403 {
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4404 if (charmaptranslate_lookup(*curinp, mapping, res))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4405 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4406 if (*res==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4407 /* not found => default to 1:1 mapping */
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4408 *(*outp)++ = *curinp;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4409 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4410 else if (*res==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4411 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4412 else if (PyInt_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4413 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4414 *(*outp)++ = (Py_UNICODE)PyInt_AS_LONG(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4415 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4416 else if (PyUnicode_Check(*res)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4417 Py_ssize_t repsize = PyUnicode_GET_SIZE(*res);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4418 if (repsize==1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4419 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4420 *(*outp)++ = *PyUnicode_AS_UNICODE(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4421 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4422 else if (repsize!=0) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4423 /* more than one character */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4424 Py_ssize_t requiredsize = (*outp-PyUnicode_AS_UNICODE(*outobj)) +
31455
37d501b9e2d5 Fix reallocation bug in unicode.translate(): The code was comparing
Walter Dörwald <walter@livinglogic.de>
parents: 31325
diff changeset
4425 (insize - (curinp-startinp)) +
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4426 repsize - 1;
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4427 if (charmaptranslate_makespace(outobj, outp, requiredsize))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4428 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4429 memcpy(*outp, PyUnicode_AS_UNICODE(*res), sizeof(Py_UNICODE)*repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4430 *outp += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4431 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4432 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4433 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4434 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4435 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4436 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4437
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4438 PyObject *PyUnicode_TranslateCharmap(const Py_UNICODE *p,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4439 Py_ssize_t size,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4440 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4441 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4442 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4443 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4444 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4445 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4446 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4447 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4448 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4449 Py_UNICODE *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4450 /* current output position */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4451 Py_ssize_t respos = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4452 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4453 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4454 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4455 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4456 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4457 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4458 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4459
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4460 if (mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4461 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4462 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4463 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4464
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4465 /* allocate enough for a simple 1:1 translation without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4466 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4467 res = PyUnicode_FromUnicode(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4468 if (res == NULL)
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4469 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4470 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4471 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4472 str = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4473
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4474 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4475 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4476 PyObject *x = NULL;
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4477 if (charmaptranslate_output(startp, p, size, mapping, &res, &str, &x)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4478 Py_XDECREF(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4479 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4480 }
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
4481 Py_XDECREF(x);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4482 if (x!=Py_None) /* it worked => adjust input pointer */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4483 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4484 else { /* untranslatable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4485 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4486 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4487 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4488 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4489 /* startpos for collecting untranslatable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4490 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4491 const Py_UNICODE *collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4492 const Py_UNICODE *coll;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4493
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4494 /* find all untranslatable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4495 while (collend < endp) {
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4496 if (charmaptranslate_lookup(*collend, mapping, &x))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4497 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4498 Py_XDECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4499 if (x!=Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4500 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4501 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4502 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4503 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4504 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4505 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4506 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4507 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4508 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4509 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4510 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4511 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4512 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4513 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4514 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4515 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4516 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4517 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4518 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4519 raise_translate_exception(&exc, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4520 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4521 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4522 /* No need to check for space, this is a 1:1 replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4523 for (coll = collstart; coll<collend; ++coll)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4524 *str++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4525 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4526 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4527 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4528 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4529 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4530 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4531 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4532 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4533 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4534 sprintf(buffer, "&#%d;", (int)*p);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4535 if (charmaptranslate_makespace(&res, &str,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4536 (str-PyUnicode_AS_UNICODE(res))+strlen(buffer)+(endp-collend)))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4537 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4538 for (cp = buffer; *cp; ++cp)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4539 *str++ = *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4540 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4541 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4542 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4543 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4544 repunicode = unicode_translate_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4545 reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4546 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4547 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4548 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4549 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4550 repsize = PyUnicode_GET_SIZE(repunicode);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4551 if (charmaptranslate_makespace(&res, &str,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4552 (str-PyUnicode_AS_UNICODE(res))+repsize+(endp-collend))) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4553 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4554 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4555 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4556 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4557 *str++ = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4558 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4559 Py_DECREF(repunicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4560 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4561 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4562 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4563 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4564 respos = str-PyUnicode_AS_UNICODE(res);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
4565 if (respos<PyUnicode_GET_SIZE(res)) {
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
4566 if (_PyUnicode_Resize(&res, respos) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4567 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4568 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4569 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4570 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4571 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4572
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4573 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4574 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4575 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4576 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4577 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4578 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4579
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4580 PyObject *PyUnicode_Translate(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4581 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4582 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4583 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4584 PyObject *result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4585
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4586 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4587 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4588 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4589 result = PyUnicode_TranslateCharmap(PyUnicode_AS_UNICODE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4590 PyUnicode_GET_SIZE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4591 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4592 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4593 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4594 return result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4595
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4596 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4597 Py_XDECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4598 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4599 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4600
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4601 /* --- Decimal Encoder ---------------------------------------------------- */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4602
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4603 int PyUnicode_EncodeDecimal(Py_UNICODE *s,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4604 Py_ssize_t length,
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4605 char *output,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4606 const char *errors)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4607 {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4608 Py_UNICODE *p, *end;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4609 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4610 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4611 const char *encoding = "decimal";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4612 const char *reason = "invalid decimal Unicode string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4613 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4614 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4615 int known_errorHandler = -1;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4616
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4617 if (output == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4618 PyErr_BadArgument();
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4619 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4620 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4621
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4622 p = s;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4623 end = s + length;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4624 while (p < end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4625 register Py_UNICODE ch = *p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4626 int decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4627 PyObject *repunicode;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4628 Py_ssize_t repsize;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4629 Py_ssize_t newpos;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4630 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4631 Py_UNICODE *collstart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4632 Py_UNICODE *collend;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4633
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4634 if (Py_UNICODE_ISSPACE(ch)) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4635 *output++ = ' ';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4636 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4637 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4638 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4639 decimal = Py_UNICODE_TODECIMAL(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4640 if (decimal >= 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4641 *output++ = '0' + decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4642 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4643 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4644 }
12275
cbfa95a6a03e Conrad Huang points out that "if (0 < ch < 256)", while legal C,
Guido van Rossum <guido@python.org>
parents: 12249
diff changeset
4645 if (0 < ch && ch < 256) {
12577
85f00f8cd674 Fix warning detected by VC++ on assignment of Py_UNICODE to char.
Guido van Rossum <guido@python.org>
parents: 12576
diff changeset
4646 *output++ = (char)ch;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4647 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4648 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4649 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4650 /* All other characters are considered unencodable */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4651 collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4652 collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4653 while (collend < end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4654 if ((0 < *collend && *collend < 256) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4655 !Py_UNICODE_ISSPACE(*collend) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4656 Py_UNICODE_TODECIMAL(*collend))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4657 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4658 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4659 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4660 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4661 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4662 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4663 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4664 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4665 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4666 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4667 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4668 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4669 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4670 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4671 known_errorHandler = 0;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4672 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4673 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4674 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4675 raise_encode_exception(&exc, encoding, s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4676 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4677 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4678 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4679 *output++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4680 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4681 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4682 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4683 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4684 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4685 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4686 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4687 output += sprintf(output, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4688 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4689 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4690 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4691 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4692 encoding, reason, s, length, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4693 collstart-s, collend-s, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4694 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4695 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4696 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4697 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4698 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4699 Py_UNICODE ch = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4700 if (Py_UNICODE_ISSPACE(ch))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4701 *output++ = ' ';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4702 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4703 decimal = Py_UNICODE_TODECIMAL(ch);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4704 if (decimal >= 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4705 *output++ = '0' + decimal;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4706 else if (0 < ch && ch < 256)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4707 *output++ = (char)ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4708 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4709 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4710 raise_encode_exception(&exc, encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4711 s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4712 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4713 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4714 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4715 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4716 p = s + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4717 Py_DECREF(repunicode);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4718 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4719 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4720 /* 0-terminate the output string */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4721 *output++ = '\0';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4722 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4723 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4724 return 0;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4725
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4726 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4727 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4728 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4729 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4730 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4731
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4732 /* --- Helpers ------------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4733
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4734 #define STRINGLIB_CHAR Py_UNICODE
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4735
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4736 #define STRINGLIB_LEN PyUnicode_GET_SIZE
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4737 #define STRINGLIB_NEW PyUnicode_FromUnicode
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4738 #define STRINGLIB_STR PyUnicode_AS_UNICODE
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4739
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4740 Py_LOCAL_INLINE(int)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4741 STRINGLIB_CMP(const Py_UNICODE* str, const Py_UNICODE* other, Py_ssize_t len)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4742 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4743 if (str[0] != other[0])
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4744 return 1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4745 return memcmp((void*) str, (void*) other, len * sizeof(Py_UNICODE));
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4746 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4747
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4748 #define STRINGLIB_EMPTY unicode_empty
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4749
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4750 #include "stringlib/fastsearch.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4751
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4752 #include "stringlib/count.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4753 #include "stringlib/find.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4754 #include "stringlib/partition.h"
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4755
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4756 /* helper macro to fixup start/end slice values */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4757 #define FIX_START_END(obj) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4758 if (start < 0) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4759 start += (obj)->length; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4760 if (start < 0) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4761 start = 0; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4762 if (end > (obj)->length) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4763 end = (obj)->length; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4764 if (end < 0) \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4765 end += (obj)->length; \
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4766 if (end < 0) \
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
4767 end = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
4768
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4769 Py_ssize_t PyUnicode_Count(PyObject *str,
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4770 PyObject *substr,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4771 Py_ssize_t start,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4772 Py_ssize_t end)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4773 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4774 Py_ssize_t result;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4775 PyUnicodeObject* str_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4776 PyUnicodeObject* sub_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4777
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4778 str_obj = (PyUnicodeObject*) PyUnicode_FromObject(str);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4779 if (!str_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4780 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4781 sub_obj = (PyUnicodeObject*) PyUnicode_FromObject(substr);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4782 if (!sub_obj) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4783 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4784 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4785 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4786
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4787 FIX_START_END(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4788
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4789 result = stringlib_count(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4790 str_obj->str + start, end - start, sub_obj->str, sub_obj->length
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4791 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4792
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4793 Py_DECREF(sub_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4794 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4795
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4796 return result;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4797 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4798
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4799 Py_ssize_t PyUnicode_Find(PyObject *str,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4800 PyObject *sub,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4801 Py_ssize_t start,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4802 Py_ssize_t end,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4803 int direction)
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4804 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4805 Py_ssize_t result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4806
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4807 str = PyUnicode_FromObject(str);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4808 if (!str)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4809 return -2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4810 sub = PyUnicode_FromObject(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4811 if (!sub) {
12957
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
4812 Py_DECREF(str);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4813 return -2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4814 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4815
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4816 if (direction > 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4817 result = stringlib_find_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4818 PyUnicode_AS_UNICODE(str), PyUnicode_GET_SIZE(str),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4819 PyUnicode_AS_UNICODE(sub), PyUnicode_GET_SIZE(sub),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4820 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4821 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4822 else
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4823 result = stringlib_rfind_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4824 PyUnicode_AS_UNICODE(str), PyUnicode_GET_SIZE(str),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4825 PyUnicode_AS_UNICODE(sub), PyUnicode_GET_SIZE(sub),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4826 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4827 );
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4828
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4829 Py_DECREF(str);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4830 Py_DECREF(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4831
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4832 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4833 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4834
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4835 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4836 int tailmatch(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4837 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4838 Py_ssize_t start,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4839 Py_ssize_t end,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4840 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4841 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4842 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4843 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4844
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
4845 FIX_START_END(self);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4846
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4847 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4848 if (end < start)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4849 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4850
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4851 if (direction > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4852 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4853 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4854 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4855 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4856 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4857 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4858
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4859 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4860 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4861
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4862 Py_ssize_t PyUnicode_Tailmatch(PyObject *str,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4863 PyObject *substr,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4864 Py_ssize_t start,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4865 Py_ssize_t end,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4866 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4867 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4868 Py_ssize_t result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4869
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4870 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4871 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4872 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4873 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4874 if (substr == NULL) {
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
4875 Py_DECREF(str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4876 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4877 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4878
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4879 result = tailmatch((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4880 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4881 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4882 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4883 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4884 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4885 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4886
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4887 /* Apply fixfct filter to the Unicode object self and return a
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4888 reference to the modified object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4889
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4890 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4891 PyObject *fixup(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4892 int (*fixfct)(PyUnicodeObject *s))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4893 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4894
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4895 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4896
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4897 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4898 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4899 return NULL;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4900
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4901 Py_UNICODE_COPY(u->str, self->str, self->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4902
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4903 if (!fixfct(u) && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4904 /* fixfct should return TRUE if it modified the buffer. If
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4905 FALSE, return a reference to the original buffer instead
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4906 (to save space, not time) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4907 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4908 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4909 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4910 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4911 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4912 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4913
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4914 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4915 int fixupper(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4916 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4917 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4918 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4919 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4920
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4921 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4922 register Py_UNICODE ch;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4923
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4924 ch = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4925 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4926 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4927 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4928 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4929 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4930 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4931
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4932 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4933 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4934
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4935 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4936 int fixlower(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4937 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4938 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4939 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4940 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4941
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4942 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4943 register Py_UNICODE ch;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4944
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4945 ch = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4946 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4947 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4948 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4949 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4950 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4951 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4952
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4953 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4954 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4955
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4956 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4957 int fixswapcase(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4958 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4959 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4960 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4961 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4962
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4963 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4964 if (Py_UNICODE_ISUPPER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4965 *s = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4966 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4967 } else if (Py_UNICODE_ISLOWER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4968 *s = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4969 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4970 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4971 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4972 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4973
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4974 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4975 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4976
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4977 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4978 int fixcapitalize(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4979 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
4980 Py_ssize_t len = self->length;
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4981 Py_UNICODE *s = self->str;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4982 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4983
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4984 if (len == 0)
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4985 return 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4986 if (Py_UNICODE_ISLOWER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4987 *s = Py_UNICODE_TOUPPER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4988 status = 1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4989 }
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4990 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4991 while (--len > 0) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4992 if (Py_UNICODE_ISUPPER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4993 *s = Py_UNICODE_TOLOWER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4994 status = 1;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4995 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4996 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4997 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4998 return status;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4999 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5000
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5001 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5002 int fixtitle(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5003 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5004 register Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5005 register Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5006 int previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5007
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5008 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5009 if (PyUnicode_GET_SIZE(self) == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5010 Py_UNICODE ch = Py_UNICODE_TOTITLE(*p);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5011 if (*p != ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5012 *p = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5013 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5014 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5015 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5016 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5017 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5018
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5019 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5020 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5021 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5022 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5023
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5024 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5025 *p = Py_UNICODE_TOLOWER(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5026 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5027 *p = Py_UNICODE_TOTITLE(ch);
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5028
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5029 if (Py_UNICODE_ISLOWER(ch) ||
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5030 Py_UNICODE_ISUPPER(ch) ||
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5031 Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5032 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5033 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5034 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5035 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5036 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5037 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5038
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5039 PyObject *
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5040 PyUnicode_Join(PyObject *separator, PyObject *seq)
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5041 {
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5042 PyObject *internal_separator = NULL;
33499
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc.
Skip Montanaro <skip@pobox.com>
parents: 33495
diff changeset
5043 const Py_UNICODE blank = ' ';
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc.
Skip Montanaro <skip@pobox.com>
parents: 33495
diff changeset
5044 const Py_UNICODE *sep = &blank;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5045 Py_ssize_t seplen = 1;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5046 PyUnicodeObject *res = NULL; /* the result */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5047 Py_ssize_t res_alloc = 100; /* # allocated bytes for string in res */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5048 Py_ssize_t res_used; /* # used bytes */
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5049 Py_UNICODE *res_p; /* pointer to free byte in res's string area */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5050 PyObject *fseq; /* PySequence_Fast(seq) */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5051 Py_ssize_t seqlen; /* len(fseq) -- number of items in sequence */
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5052 PyObject *item;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5053 Py_ssize_t i;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5054
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5055 fseq = PySequence_Fast(seq, "");
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5056 if (fseq == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5057 return NULL;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5058 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5059
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5060 /* Grrrr. A codec may be invoked to convert str objects to
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5061 * Unicode, and so it's possible to call back into Python code
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5062 * during PyUnicode_FromObject(), and so it's possible for a sick
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5063 * codec to change the size of fseq (if seq is a list). Therefore
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5064 * we have to keep refetching the size -- can't assume seqlen
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5065 * is invariant.
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5066 */
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5067 seqlen = PySequence_Fast_GET_SIZE(fseq);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5068 /* If empty sequence, return u"". */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5069 if (seqlen == 0) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5070 res = _PyUnicode_New(0); /* empty sequence; return u"" */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5071 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5072 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5073 /* If singleton sequence with an exact Unicode, return that. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5074 if (seqlen == 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5075 item = PySequence_Fast_GET_ITEM(fseq, 0);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5076 if (PyUnicode_CheckExact(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5077 Py_INCREF(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5078 res = (PyUnicodeObject *)item;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5079 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5080 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5081 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5082
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5083 /* At least two items to join, or one that isn't exact Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5084 if (seqlen > 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5085 /* Set up sep and seplen -- they're needed. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5086 if (separator == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5087 sep = &blank;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5088 seplen = 1;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5089 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5090 else {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5091 internal_separator = PyUnicode_FromObject(separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5092 if (internal_separator == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5093 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5094 sep = PyUnicode_AS_UNICODE(internal_separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5095 seplen = PyUnicode_GET_SIZE(internal_separator);
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5096 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5097 seqlen = PySequence_Fast_GET_SIZE(fseq);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5098 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5099 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5100
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5101 /* Get space. */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5102 res = _PyUnicode_New(res_alloc);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5103 if (res == NULL)
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5104 goto onError;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5105 res_p = PyUnicode_AS_UNICODE(res);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5106 res_used = 0;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5107
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5108 for (i = 0; i < seqlen; ++i) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5109 Py_ssize_t itemlen;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5110 Py_ssize_t new_res_used;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5111
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5112 item = PySequence_Fast_GET_ITEM(fseq, i);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5113 /* Convert item to Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5114 if (! PyUnicode_Check(item) && ! PyString_Check(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5115 PyErr_Format(PyExc_TypeError,
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5116 "sequence item %zd: expected string or Unicode,"
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5117 " %.80s found",
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5118 i, item->ob_type->tp_name);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
5119 goto onError;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
5120 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5121 item = PyUnicode_FromObject(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5122 if (item == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5123 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5124 /* We own a reference to item from here on. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5125
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5126 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5127 seqlen = PySequence_Fast_GET_SIZE(fseq);
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
5128
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5129 /* Make sure we have enough space for the separator and the item. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5130 itemlen = PyUnicode_GET_SIZE(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5131 new_res_used = res_used + itemlen;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
5132 if (new_res_used < 0)
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5133 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5134 if (i < seqlen - 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5135 new_res_used += seplen;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
5136 if (new_res_used < 0)
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5137 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5138 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5139 if (new_res_used > res_alloc) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5140 /* double allocated size until it's big enough */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5141 do {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5142 res_alloc += res_alloc;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5143 if (res_alloc <= 0)
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5144 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5145 } while (new_res_used > res_alloc);
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5146 if (_PyUnicode_Resize(&res, res_alloc) < 0) {
20007
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
5147 Py_DECREF(item);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
5148 goto onError;
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
5149 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5150 res_p = PyUnicode_AS_UNICODE(res) + res_used;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5151 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5152
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5153 /* Copy item, and maybe the separator. */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5154 Py_UNICODE_COPY(res_p, PyUnicode_AS_UNICODE(item), itemlen);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5155 res_p += itemlen;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5156 if (i < seqlen - 1) {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5157 Py_UNICODE_COPY(res_p, sep, seplen);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5158 res_p += seplen;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5159 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5160 Py_DECREF(item);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5161 res_used = new_res_used;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5162 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5163
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5164 /* Shrink res to match the used area; this probably can't fail,
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5165 * but it's cheap to check.
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5166 */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5167 if (_PyUnicode_Resize(&res, res_used) < 0)
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5168 goto onError;
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5169
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5170 Done:
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5171 Py_XDECREF(internal_separator);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5172 Py_DECREF(fseq);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5173 return (PyObject *)res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5174
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5175 Overflow:
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5176 PyErr_SetString(PyExc_OverflowError,
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
5177 "join() result is too long for a Python string");
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5178 Py_DECREF(item);
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5179 /* fall through */
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5180
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5181 onError:
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
5182 Py_XDECREF(internal_separator);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
5183 Py_DECREF(fseq);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
5184 Py_XDECREF(res);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5185 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5186 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5187
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5188 static
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5189 PyUnicodeObject *pad(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5190 Py_ssize_t left,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5191 Py_ssize_t right,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5192 Py_UNICODE fill)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5193 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5194 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5195
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5196 if (left < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5197 left = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5198 if (right < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5199 right = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5200
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5201 if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5202 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5203 return self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5204 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5205
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5206 u = _PyUnicode_New(left + self->length + right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5207 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5208 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5209 Py_UNICODE_FILL(u->str, fill, left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5210 Py_UNICODE_COPY(u->str + left, self->str, self->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5211 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5212 Py_UNICODE_FILL(u->str + left + self->length, fill, right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5213 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5214
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5215 return u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5216 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5217
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5218 #define SPLIT_APPEND(data, left, right) \
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5219 str = PyUnicode_FromUnicode((data) + (left), (right) - (left)); \
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5220 if (!str) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5221 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5222 if (PyList_Append(list, str)) { \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5223 Py_DECREF(str); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5224 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5225 } \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5226 else \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5227 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5228
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5229 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5230 PyObject *split_whitespace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5231 PyObject *list,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5232 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5233 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5234 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5235 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5236 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5237 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5239 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5240 /* find a token */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5241 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5242 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5243 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5244 while (i < len && !Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5245 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5246 if (j < i) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5247 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5248 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5249 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5250 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5251 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5252 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5253 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5254 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5255 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5256 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5257 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5258 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5259
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5260 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5261 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5262 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5264
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5265 PyObject *PyUnicode_Splitlines(PyObject *string,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5266 int keepends)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5267 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5268 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5269 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5270 Py_ssize_t len;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5271 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5272 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5273 Py_UNICODE *data;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5274
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5275 string = PyUnicode_FromObject(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5276 if (string == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5277 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5278 data = PyUnicode_AS_UNICODE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5279 len = PyUnicode_GET_SIZE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5280
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5281 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5282 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5283 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5284
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5285 for (i = j = 0; i < len; ) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5286 Py_ssize_t eol;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5287
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5288 /* Find a line and append it */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5289 while (i < len && !BLOOM_LINEBREAK(data[i]))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5290 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5291
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5292 /* Skip the line break reading CRLF as one line break */
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5293 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5294 if (i < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5295 if (data[i] == '\r' && i + 1 < len &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5296 data[i+1] == '\n')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5297 i += 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5298 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5299 i++;
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5300 if (keepends)
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5301 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5302 }
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5303 SPLIT_APPEND(data, j, eol);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5304 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5305 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5306 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5307 SPLIT_APPEND(data, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5308 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5309
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5310 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5311 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5312
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5313 onError:
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
5314 Py_XDECREF(list);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5315 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5316 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5317 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5318
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5319 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5320 PyObject *split_char(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5321 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5322 Py_UNICODE ch,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5323 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5324 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5325 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5326 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5327 Py_ssize_t len = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5328 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5329
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5330 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5331 if (self->str[i] == ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5332 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5333 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5334 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5335 i = j = i + 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5336 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5337 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5338 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5339 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5340 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5341 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5342 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5343
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5344 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5345 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5346 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5347 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5348
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5349 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5350 PyObject *split_substring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5351 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5352 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5353 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5354 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5355 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5356 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5357 Py_ssize_t len = self->length;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5358 Py_ssize_t sublen = substring->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5359 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5360
15609
0906c8e2c961 Fix off-by-one error in split_substring(). Fixes SF bug #122162.
Guido van Rossum <guido@python.org>
parents: 15579
diff changeset
5361 for (i = j = 0; i <= len - sublen; ) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5362 if (Py_UNICODE_MATCH(self, i, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5363 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5364 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5365 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5366 i = j = i + sublen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5367 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5368 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5369 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5370 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5371 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5372 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5373 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5374
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5375 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5376 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5377 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5378 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5379
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5380 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5381 PyObject *rsplit_whitespace(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5382 PyObject *list,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5383 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5384 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5385 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5386 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5387 Py_ssize_t len = self->length;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5388 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5389
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5390 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5391 /* find a token */
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5392 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5393 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5394 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5395 while (i >= 0 && !Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5396 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5397 if (j > i) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5398 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5399 break;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5400 SPLIT_APPEND(self->str, i + 1, j + 1);
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5401 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5402 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5403 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5404 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5405 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5406 if (j >= 0) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5407 SPLIT_APPEND(self->str, 0, j + 1);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5408 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5409 if (PyList_Reverse(list) < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5410 goto onError;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5411 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5412
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5413 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5414 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5415 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5416 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5417
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5418 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5419 PyObject *rsplit_char(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5420 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5421 Py_UNICODE ch,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5422 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5423 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5424 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5425 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5426 Py_ssize_t len = self->length;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5427 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5428
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5429 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5430 if (self->str[i] == ch) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5431 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5432 break;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5433 SPLIT_APPEND(self->str, i + 1, j + 1);
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5434 j = i = i - 1;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5435 } else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5436 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5437 }
31252
892116a886e0 Fix unicode.rsplit()'s bug that ignores separater on the end of string when
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31248
diff changeset
5438 if (j >= -1) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5439 SPLIT_APPEND(self->str, 0, j + 1);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5440 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5441 if (PyList_Reverse(list) < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5442 goto onError;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5443 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5444
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5445 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5446 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5447 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5448 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5449
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5450 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5451 PyObject *rsplit_substring(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5452 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5453 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5454 Py_ssize_t maxcount)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5455 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5456 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5457 register Py_ssize_t j;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5458 Py_ssize_t len = self->length;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5459 Py_ssize_t sublen = substring->length;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5460 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5461
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5462 for (i = len - sublen, j = len; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5463 if (Py_UNICODE_MATCH(self, i, substring)) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5464 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5465 break;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5466 SPLIT_APPEND(self->str, i + sublen, j);
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5467 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5468 i -= sublen;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5469 } else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5470 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5471 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5472 if (j >= 0) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5473 SPLIT_APPEND(self->str, 0, j);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5474 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5475 if (PyList_Reverse(list) < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5476 goto onError;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5477 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5478
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5479 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5480 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5481 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5482 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5483
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5484 #undef SPLIT_APPEND
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5485
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5486 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5487 PyObject *split(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5488 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5489 Py_ssize_t maxcount)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5490 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5491 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5492
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5493 if (maxcount < 0)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5494 maxcount = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5495
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5496 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5497 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5498 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5499
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5500 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5501 return split_whitespace(self,list,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5502
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5503 else if (substring->length == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5504 return split_char(self,list,substring->str[0],maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5505
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5506 else if (substring->length == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5507 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5508 PyErr_SetString(PyExc_ValueError, "empty separator");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5509 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5510 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5511 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5512 return split_substring(self,list,substring,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5513 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5514
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5515 static
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5516 PyObject *rsplit(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5517 PyUnicodeObject *substring,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5518 Py_ssize_t maxcount)
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5519 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5520 PyObject *list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5521
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5522 if (maxcount < 0)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5523 maxcount = PY_SSIZE_T_MAX;
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5524
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5525 list = PyList_New(0);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5526 if (!list)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5527 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5528
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5529 if (substring == NULL)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5530 return rsplit_whitespace(self,list,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5531
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5532 else if (substring->length == 1)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5533 return rsplit_char(self,list,substring->str[0],maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5534
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5535 else if (substring->length == 0) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5536 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5537 PyErr_SetString(PyExc_ValueError, "empty separator");
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5538 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5539 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5540 else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5541 return rsplit_substring(self,list,substring,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5542 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5543
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
5544 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5545 PyObject *replace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5546 PyUnicodeObject *str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5547 PyUnicodeObject *str2,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5548 Py_ssize_t maxcount)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5549 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5550 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5551
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5552 if (maxcount < 0)
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5553 maxcount = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5554
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5555 if (str1->length == str2->length) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5556 /* same length */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
5557 Py_ssize_t i;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5558 if (str1->length == 1) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5559 /* replace characters */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5560 Py_UNICODE u1, u2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5561 if (!findchar(self->str, self->length, str1->str[0]))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5562 goto nothing;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5563 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5564 if (!u)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5565 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5566 Py_UNICODE_COPY(u->str, self->str, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5567 u1 = str1->str[0];
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5568 u2 = str2->str[0];
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5569 for (i = 0; i < u->length; i++)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5570 if (u->str[i] == u1) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5571 if (--maxcount < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5572 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5573 u->str[i] = u2;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5574 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5575 } else {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5576 i = fastsearch(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5577 self->str, self->length, str1->str, str1->length, FAST_SEARCH
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5578 );
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5579 if (i < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5580 goto nothing;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5581 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5582 if (!u)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5583 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5584 Py_UNICODE_COPY(u->str, self->str, self->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5585 while (i <= self->length - str1->length)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5586 if (Py_UNICODE_MATCH(self, i, str1)) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5587 if (--maxcount < 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5588 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5589 Py_UNICODE_COPY(u->str+i, str2->str, str2->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5590 i += str1->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5591 } else
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5592 i++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5593 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5594 } else {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5595
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5596 Py_ssize_t n, i, j, e;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5597 Py_ssize_t product, new_size, delta;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5598 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5599
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5600 /* replace strings */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5601 n = stringlib_count(self->str, self->length, str1->str, str1->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5602 if (n > maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5603 n = maxcount;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5604 if (n == 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5605 goto nothing;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5606 /* new_size = self->length + n * (str2->length - str1->length)); */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5607 delta = (str2->length - str1->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5608 if (delta == 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5609 new_size = self->length;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5610 } else {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5611 product = n * (str2->length - str1->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5612 if ((product / (str2->length - str1->length)) != n) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5613 PyErr_SetString(PyExc_OverflowError,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5614 "replace string is too long");
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5615 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5616 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5617 new_size = self->length + product;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5618 if (new_size < 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5619 PyErr_SetString(PyExc_OverflowError,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5620 "replace string is too long");
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5621 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5622 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5623 }
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5624 u = _PyUnicode_New(new_size);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5625 if (!u)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5626 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5627 i = 0;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5628 p = u->str;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5629 e = self->length - str1->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5630 if (str1->length > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5631 while (n-- > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5632 /* look for next match */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5633 j = i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5634 while (j <= e) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5635 if (Py_UNICODE_MATCH(self, j, str1))
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5636 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5637 j++;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5638 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5639 if (j > i) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5640 if (j > e)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5641 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5642 /* copy unchanged part [i:j] */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5643 Py_UNICODE_COPY(p, self->str+i, j-i);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5644 p += j - i;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5645 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5646 /* copy substitution string */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5647 if (str2->length > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5648 Py_UNICODE_COPY(p, str2->str, str2->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5649 p += str2->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5650 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5651 i = j + str1->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5652 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5653 if (i < self->length)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5654 /* copy tail [i:] */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5655 Py_UNICODE_COPY(p, self->str+i, self->length-i);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5656 } else {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5657 /* interleave */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5658 while (n > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5659 Py_UNICODE_COPY(p, str2->str, str2->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5660 p += str2->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5661 if (--n <= 0)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5662 break;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5663 *p++ = self->str[i++];
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5664 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5665 Py_UNICODE_COPY(p, self->str+i, self->length-i);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5666 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5667 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5668 return (PyObject *) u;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5669
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5670 nothing:
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5671 /* nothing to replace; return original string (when possible) */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5672 if (PyUnicode_CheckExact(self)) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5673 Py_INCREF(self);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5674 return (PyObject *) self;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5675 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5676 return PyUnicode_FromUnicode(self->str, self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5677 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5678
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5679 /* --- Unicode Object Methods --------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5680
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5681 PyDoc_STRVAR(title__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5682 "S.title() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5683 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5684 Return a titlecased version of S, i.e. words start with title case\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5685 characters, all remaining cased characters have lower case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5686
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5687 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5688 unicode_title(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5689 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5690 return fixup(self, fixtitle);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5691 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5692
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5693 PyDoc_STRVAR(capitalize__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5694 "S.capitalize() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5695 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5696 Return a capitalized version of S, i.e. make the first character\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5697 have upper case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5698
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5699 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5700 unicode_capitalize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5701 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5702 return fixup(self, fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5703 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5704
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5705 #if 0
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5706 PyDoc_STRVAR(capwords__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5707 "S.capwords() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5708 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5709 Apply .capitalize() to all words in S and return the result with\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5710 normalized whitespace (all whitespace strings are replaced by ' ').");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5711
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5712 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5713 unicode_capwords(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5714 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5715 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5716 PyObject *item;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5717 Py_ssize_t i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5718
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5719 /* Split into words */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5720 list = split(self, NULL, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5721 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5722 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5723
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5724 /* Capitalize each word */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5725 for (i = 0; i < PyList_GET_SIZE(list); i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5726 item = fixup((PyUnicodeObject *)PyList_GET_ITEM(list, i),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5727 fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5728 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5729 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5730 Py_DECREF(PyList_GET_ITEM(list, i));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5731 PyList_SET_ITEM(list, i, item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5732 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5733
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5734 /* Join the words to form a new string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5735 item = PyUnicode_Join(NULL, list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5736
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5737 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5738 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5739 return (PyObject *)item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5740 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5741 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5742
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5743 /* Argument converter. Coerces to a single unicode character */
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5744
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5745 static int
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5746 convert_uc(PyObject *obj, void *addr)
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5747 {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5748 Py_UNICODE *fillcharloc = (Py_UNICODE *)addr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5749 PyObject *uniobj;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5750 Py_UNICODE *unistr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5751
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5752 uniobj = PyUnicode_FromObject(obj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5753 if (uniobj == NULL) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5754 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5755 "The fill character cannot be converted to Unicode");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5756 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5757 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5758 if (PyUnicode_GET_SIZE(uniobj) != 1) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5759 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5760 "The fill character must be exactly one character long");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5761 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5762 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5763 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5764 unistr = PyUnicode_AS_UNICODE(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5765 *fillcharloc = unistr[0];
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5766 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5767 return 1;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5768 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5769
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5770 PyDoc_STRVAR(center__doc__,
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5771 "S.center(width[, fillchar]) -> unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5772 \n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5773 Return S centered in a Unicode string of length width. Padding is\n\
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5774 done using the specified fill character (default is a space)");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5775
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5776 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5777 unicode_center(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5778 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5779 Py_ssize_t marg, left;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5780 Py_ssize_t width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5781 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5782
36532
0c096be27000 Use correct PyArg_Parse format char for Py_ssize_t in unicode.center().
Thomas Wouters <thomas@python.org>
parents: 36519
diff changeset
5783 if (!PyArg_ParseTuple(args, "n|O&:center", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5784 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5785
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5786 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5787 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5788 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5789 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5790
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5791 marg = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5792 left = marg / 2 + (marg & width & 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5793
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5794 return (PyObject*) pad(self, left, marg - left, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5795 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5796
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5797 #if 0
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5798
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5799 /* This code should go into some future Unicode collation support
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5800 module. The basic comparison should compare ordinals on a naive
13946
7a99450da2d6 Clean up warning from Monterey compiler.
Trent Mick <trentm@activestate.com>
parents: 13890
diff changeset
5801 basis (this is what Java does and thus JPython too). */
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5802
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5803 /* speedy UTF-16 code point order comparison */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5804 /* gleaned from: */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5805 /* http://www-4.ibm.com/software/developer/library/utf16.html?dwzone=unicode */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5806
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
5807 static short utf16Fixup[32] =
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5808 {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5809 0, 0, 0, 0, 0, 0, 0, 0,
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5810 0, 0, 0, 0, 0, 0, 0, 0,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5811 0, 0, 0, 0, 0, 0, 0, 0,
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
5812 0, 0, 0, 0x2000, -0x800, -0x800, -0x800, -0x800
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5813 };
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5814
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5815 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5816 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5817 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5818 Py_ssize_t len1, len2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5819
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5820 Py_UNICODE *s1 = str1->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5821 Py_UNICODE *s2 = str2->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5822
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5823 len1 = str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5824 len2 = str2->length;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5825
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5826 while (len1 > 0 && len2 > 0) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5827 Py_UNICODE c1, c2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5828
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5829 c1 = *s1++;
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5830 c2 = *s2++;
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5831
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5832 if (c1 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5833 c1 += utf16Fixup[c1>>11];
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5834 if (c2 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5835 c2 += utf16Fixup[c2>>11];
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5836 /* now c1 and c2 are in UTF-32-compatible order */
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5837
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5838 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5839 return (c1 < c2) ? -1 : 1;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5840
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
5841 len1--; len2--;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5842 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5843
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5844 return (len1 < len2) ? -1 : (len1 != len2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5845 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5846
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5847 #else
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5848
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5849 static int
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5850 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5851 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5852 register Py_ssize_t len1, len2;
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5853
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5854 Py_UNICODE *s1 = str1->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5855 Py_UNICODE *s2 = str2->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5856
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5857 len1 = str1->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5858 len2 = str2->length;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5859
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5860 while (len1 > 0 && len2 > 0) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5861 Py_UNICODE c1, c2;
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5862
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5863 c1 = *s1++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5864 c2 = *s2++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5865
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5866 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5867 return (c1 < c2) ? -1 : 1;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
5868
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5869 len1--; len2--;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5870 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5871
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5872 return (len1 < len2) ? -1 : (len1 != len2);
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5873 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5874
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5875 #endif
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
5876
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5877 int PyUnicode_Compare(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5878 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5879 {
41596
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5880 if (PyUnicode_Check(left) && PyUnicode_Check(right))
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5881 return unicode_compare((PyUnicodeObject *)left,
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5882 (PyUnicodeObject *)right);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5883 if ((PyString_Check(left) && PyUnicode_Check(right)) ||
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5884 (PyUnicode_Check(left) && PyString_Check(right))) {
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5885 if (PyUnicode_Check(left))
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5886 left = _PyUnicode_AsDefaultEncodedString(left, NULL);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5887 if (PyUnicode_Check(right))
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5888 right = _PyUnicode_AsDefaultEncodedString(right, NULL);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5889 assert(PyString_Check(left));
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5890 assert(PyString_Check(right));
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5891 return PyObject_Compare(left, right);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5892 }
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5893 PyErr_Format(PyExc_TypeError,
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5894 "Can't compare %.100s and %.100s",
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5895 left->ob_type->tp_name,
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
5896 right->ob_type->tp_name);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5897 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5898 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5899
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5900 int
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5901 PyUnicode_CompareWithASCIIString(PyObject* uni, const char* str)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5902 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5903 int i;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5904 Py_UNICODE *id;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5905 assert(PyUnicode_Check(uni));
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5906 id = PyUnicode_AS_UNICODE(uni);
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5907 /* Compare Unicode string and source character set string */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5908 for (i = 0; id[i] && str[i]; i++)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5909 if (id[i] != str[i])
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5910 return ((int)id[i] < (int)str[i]) ? -1 : 1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5911 if (id[i])
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5912 return 1; /* uni is longer */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5913 if (str[i])
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5914 return -1; /* str is longer */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5915 return 0;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5916 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
5917
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5918 PyObject *PyUnicode_RichCompare(PyObject *left,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5919 PyObject *right,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5920 int op)
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5921 {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5922 int result;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5923
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5924 result = PyUnicode_Compare(left, right);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5925 if (result == -1 && PyErr_Occurred())
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5926 goto onError;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5927
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5928 /* Convert the return value to a Boolean */
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5929 switch (op) {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5930 case Py_EQ:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5931 result = (result == 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5932 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5933 case Py_NE:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5934 result = (result != 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5935 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5936 case Py_LE:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5937 result = (result <= 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5938 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5939 case Py_GE:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5940 result = (result >= 0);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5941 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5942 case Py_LT:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5943 result = (result == -1);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5944 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5945 case Py_GT:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5946 result = (result == 1);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5947 break;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5948 }
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5949 return PyBool_FromLong(result);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5950
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5951 onError:
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5952
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5953 /* Standard case
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5954
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5955 Type errors mean that PyUnicode_FromObject() could not convert
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5956 one of the arguments (usually the right hand side) to Unicode,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5957 ie. we can't handle the comparison request. However, it is
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5958 possible that the other object knows a comparison method, which
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5959 is why we return Py_NotImplemented to give the other object a
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5960 chance.
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5961
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5962 */
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5963 if (PyErr_ExceptionMatches(PyExc_TypeError)) {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5964 PyErr_Clear();
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5965 Py_INCREF(Py_NotImplemented);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5966 return Py_NotImplemented;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5967 }
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5968 if (op != Py_EQ && op != Py_NE)
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5969 return NULL;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5970
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5971 /* Equality comparison.
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5972
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5973 This is a special case: we silence any PyExc_UnicodeDecodeError
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5974 and instead turn it into a PyErr_UnicodeWarning.
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5975
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5976 */
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5977 if (!PyErr_ExceptionMatches(PyExc_UnicodeDecodeError))
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5978 return NULL;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5979 PyErr_Clear();
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5980 if (PyErr_Warn(PyExc_UnicodeWarning,
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5981 (op == Py_EQ) ?
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5982 "Unicode equal comparison "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5983 "failed to convert both arguments to Unicode - "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5984 "interpreting them as being unequal" :
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5985 "Unicode unequal comparison "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5986 "failed to convert both arguments to Unicode - "
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5987 "interpreting them as being unequal"
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5988 ) < 0)
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5989 return NULL;
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5990 result = (op == Py_NE);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5991 return PyBool_FromLong(result);
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5992 }
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
5993
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5994 int PyUnicode_Contains(PyObject *container,
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5995 PyObject *element)
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5996 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
5997 PyObject *str, *sub;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
5998 int result;
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5999
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6000 /* Coerce the two arguments */
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6001 sub = PyUnicode_FromObject(element);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6002 if (!sub) {
42065
4877008074a9 Mention name of left operand, if "foo in unicode_string"
Walter Dörwald <walter@livinglogic.de>
parents: 42047
diff changeset
6003 PyErr_Format(PyExc_TypeError,
4877008074a9 Mention name of left operand, if "foo in unicode_string"
Walter Dörwald <walter@livinglogic.de>
parents: 42047
diff changeset
6004 "'in <string>' requires string as left operand, not %s",
4877008074a9 Mention name of left operand, if "foo in unicode_string"
Walter Dörwald <walter@livinglogic.de>
parents: 42047
diff changeset
6005 element->ob_type->tp_name);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6006 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6007 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6008
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6009 str = PyUnicode_FromObject(container);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6010 if (!str) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6011 Py_DECREF(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6012 return -1;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6013 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6014
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6015 result = stringlib_contains_obj(str, sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6016
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6017 Py_DECREF(str);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6018 Py_DECREF(sub);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6019
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6020 return result;
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6021 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6022
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6023 /* Concat to string or Unicode object giving a new Unicode object. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6025 PyObject *PyUnicode_Concat(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6026 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6027 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6028 PyUnicodeObject *u = NULL, *v = NULL, *w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6029
41451
e2d7a4c39360 Disallow u"..." + b"..." and b"..." + u"...".
Guido van Rossum <guido@python.org>
parents: 40909
diff changeset
6030 if (PyBytes_Check(left) || PyBytes_Check(right))
e2d7a4c39360 Disallow u"..." + b"..." and b"..." + u"...".
Guido van Rossum <guido@python.org>
parents: 40909
diff changeset
6031 return PyBytes_Concat(left, right);
e2d7a4c39360 Disallow u"..." + b"..." and b"..." + u"...".
Guido van Rossum <guido@python.org>
parents: 40909
diff changeset
6032
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6033 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6034 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6035 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6036 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6037 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6038 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6039 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6040
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6041 /* Shortcuts */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6042 if (v == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6043 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6044 return (PyObject *)u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6045 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6046 if (u == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6047 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6048 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6049 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6050
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6051 /* Concat the two Unicode strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6052 w = _PyUnicode_New(u->length + v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6053 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6054 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6055 Py_UNICODE_COPY(w->str, u->str, u->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6056 Py_UNICODE_COPY(w->str + u->length, v->str, v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6057
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6058 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6059 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6060 return (PyObject *)w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6061
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6062 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6063 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6064 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6065 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6066 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6067
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6068 void
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6069 PyUnicode_Append(PyObject **pleft, PyObject *right)
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6070 {
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6071 PyObject *new;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6072 if (*pleft == NULL)
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6073 return;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6074 if (right == NULL || !PyUnicode_Check(*pleft)) {
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6075 Py_DECREF(*pleft);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6076 *pleft = NULL;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6077 return;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6078 }
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6079 new = PyUnicode_Concat(*pleft, right);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6080 Py_DECREF(*pleft);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6081 *pleft = new;
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6082 }
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6083
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6084 void
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6085 PyUnicode_AppendAndDel(PyObject **pleft, PyObject *right)
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6086 {
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6087 PyUnicode_Append(pleft, right);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6088 Py_XDECREF(right);
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6089 }
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
6090
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6091 PyDoc_STRVAR(count__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6092 "S.count(sub[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6093 \n\
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6094 Return the number of non-overlapping occurrences of substring sub in\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6095 Unicode string S[start:end]. Optional arguments start and end are\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6096 interpreted as in slice notation.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6097
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6098 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6099 unicode_count(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6100 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6101 PyUnicodeObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6102 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6103 Py_ssize_t end = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6104 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6105
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6106 if (!PyArg_ParseTuple(args, "O|O&O&:count", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6107 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6108 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6109
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6110 substring = (PyUnicodeObject *)PyUnicode_FromObject(
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6111 (PyObject *)substring);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6112 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6113 return NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6114
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6115 FIX_START_END(self);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6116
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6117 result = PyInt_FromSsize_t(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6118 stringlib_count(self->str + start, end - start,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6119 substring->str, substring->length)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6120 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6121
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6122 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6123
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6124 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6125 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6126
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6127 PyDoc_STRVAR(encode__doc__,
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6128 "S.encode([encoding[,errors]]) -> string or unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6129 \n\
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6130 Encodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6131 to the default encoding. errors may be given to set a different error\n\
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6132 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
6133 a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
6134 'xmlcharrefreplace' as well as any other name registered with\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
6135 codecs.register_error that can handle UnicodeEncodeErrors.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6136
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6137 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6138 unicode_encode(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6139 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6140 char *encoding = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6141 char *errors = NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6142 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6143
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6144 if (!PyArg_ParseTuple(args, "|ss:encode", &encoding, &errors))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6145 return NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6146 v = PyUnicode_AsEncodedObject((PyObject *)self, encoding, errors);
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6147 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6148 goto onError;
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
6149 if (!PyBytes_Check(v)) {
41598
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6150 if (PyString_Check(v)) {
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6151 /* Old codec, turn it into bytes */
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6152 PyObject *b = PyBytes_FromObject(v);
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6153 Py_DECREF(v);
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6154 return b;
ca448c607b14 Make all of test_bytes pass (except pickling, which is too badly busted).
Guido van Rossum <guido@python.org>
parents: 41596
diff changeset
6155 }
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6156 PyErr_Format(PyExc_TypeError,
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
6157 "encoder did not return a bytes object "
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6158 "(type=%.400s)",
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6159 v->ob_type->tp_name);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6160 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6161 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6162 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6163 return v;
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6164
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6165 onError:
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6166 return NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6167 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6168
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6169 PyDoc_STRVAR(decode__doc__,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6170 "S.decode([encoding[,errors]]) -> string or unicode\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6171 \n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6172 Decodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6173 to the default encoding. errors may be given to set a different error\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6174 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6175 a UnicodeDecodeError. Other possible values are 'ignore' and 'replace'\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6176 as well as any other name registerd with codecs.register_error that is\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6177 able to handle UnicodeDecodeErrors.");
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6178
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6179 static PyObject *
32571
bec0c7618625 Fix a copy&paste typo.
Marc-André Lemburg <mal@egenix.com>
parents: 32552
diff changeset
6180 unicode_decode(PyUnicodeObject *self, PyObject *args)
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6181 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6182 char *encoding = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6183 char *errors = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6184 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6185
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6186 if (!PyArg_ParseTuple(args, "|ss:decode", &encoding, &errors))
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6187 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6188 v = PyUnicode_AsDecodedObject((PyObject *)self, encoding, errors);
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6189 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6190 goto onError;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6191 if (!PyString_Check(v) && !PyUnicode_Check(v)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6192 PyErr_Format(PyExc_TypeError,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6193 "decoder did not return a string/unicode object "
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6194 "(type=%.400s)",
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6195 v->ob_type->tp_name);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6196 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6197 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6198 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6199 return v;
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6200
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6201 onError:
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
6202 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6203 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6204
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6205 PyDoc_STRVAR(expandtabs__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6206 "S.expandtabs([tabsize]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6207 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6208 Return a copy of S where all tab characters are expanded using spaces.\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6209 If tabsize is not given, a tab size of 8 characters is assumed.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6210
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6211 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6212 unicode_expandtabs(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6213 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6214 Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6215 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6216 Py_UNICODE *q;
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6217 Py_ssize_t i, j, old_j;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6218 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6219 int tabsize = 8;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6220
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6221 if (!PyArg_ParseTuple(args, "|i:expandtabs", &tabsize))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6222 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6223
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13576
diff changeset
6224 /* First pass: determine size of output string */
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6225 i = j = old_j = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6226 e = self->str + self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6227 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6228 if (*p == '\t') {
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6229 if (tabsize > 0) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6230 j += tabsize - (j % tabsize);
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6231 if (old_j > j) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6232 PyErr_SetString(PyExc_OverflowError,
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6233 "new string is too long");
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6234 return NULL;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6235 }
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6236 old_j = j;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6237 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6238 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6239 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6240 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6241 if (*p == '\n' || *p == '\r') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6242 i += j;
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6243 old_j = j = 0;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6244 if (i < 0) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6245 PyErr_SetString(PyExc_OverflowError,
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6246 "new string is too long");
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6247 return NULL;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6248 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6249 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6250 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6251
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6252 if ((i + j) < 0) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6253 PyErr_SetString(PyExc_OverflowError, "new string is too long");
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6254 return NULL;
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6255 }
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
6256
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6257 /* Second pass: create output string and fill it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6258 u = _PyUnicode_New(i + j);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6259 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6260 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6261
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6262 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6263 q = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6264
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6265 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6266 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6267 if (tabsize > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6268 i = tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6269 j += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6270 while (i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6271 *q++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6272 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6273 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6274 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6275 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6276 *q++ = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6277 if (*p == '\n' || *p == '\r')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6278 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6279 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6280
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6281 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6282 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6283
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6284 PyDoc_STRVAR(find__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6285 "S.find(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6286 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6287 Return the lowest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6288 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6289 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6290 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6291 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6292
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6293 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6294 unicode_find(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6295 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6296 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6297 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6298 Py_ssize_t end = PY_SSIZE_T_MAX;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6299 Py_ssize_t result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6300
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6301 if (!PyArg_ParseTuple(args, "O|O&O&:find", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6302 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6303 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6304 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6305 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6306 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6307
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6308 result = stringlib_find_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6309 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6310 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6311 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6312 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6313
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6314 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6315
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6316 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6317 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6318
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6319 static PyObject *
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6320 unicode_getitem(PyUnicodeObject *self, Py_ssize_t index)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6321 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6322 if (index < 0 || index >= self->length) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6323 PyErr_SetString(PyExc_IndexError, "string index out of range");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6324 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6325 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6326
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6327 return (PyObject*) PyUnicode_FromUnicode(&self->str[index], 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6328 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6329
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6330 static long
41596
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6331 unicode_hash(PyObject *self)
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6332 {
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6333 /* Since Unicode objects compare equal to their UTF-8 string
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6334 counterparts, we hash the UTF-8 string. */
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6335 PyObject *v = _PyUnicode_AsDefaultEncodedString(self, NULL);
cfb3e0021b3b Compare and hash unicode objects like their UTF-8 representations.
Guido van Rossum <guido@python.org>
parents: 41595
diff changeset
6336 return PyObject_Hash(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6337 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6338
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6339 PyDoc_STRVAR(index__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6340 "S.index(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6341 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6342 Like S.find() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6343
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6344 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6345 unicode_index(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6346 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6347 Py_ssize_t result;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6348 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6349 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6350 Py_ssize_t end = PY_SSIZE_T_MAX;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6351
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6352 if (!PyArg_ParseTuple(args, "O|O&O&:index", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6353 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6354 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6355 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6356 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6357 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6358
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6359 result = stringlib_find_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6360 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6361 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6362 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6363 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6364
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6365 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6366
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6367 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6368 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6369 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6370 }
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6371
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6372 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6373 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6374
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6375 PyDoc_STRVAR(islower__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6376 "S.islower() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6377 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6378 Return True if all cased characters in S are lowercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6379 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6380
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6381 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6382 unicode_islower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6383 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6384 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6385 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6386 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6387
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6388 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6389 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6390 return PyBool_FromLong(Py_UNICODE_ISLOWER(*p));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6391
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6392 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6393 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6394 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6395
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6396 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6397 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6398 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6399 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6400
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6401 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6402 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6403 else if (!cased && Py_UNICODE_ISLOWER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6404 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6405 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6406 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6407 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6408
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6409 PyDoc_STRVAR(isupper__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6410 "S.isupper() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6411 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6412 Return True if all cased characters in S are uppercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6413 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6414
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6415 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6416 unicode_isupper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6417 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6418 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6419 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6420 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6421
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6422 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6423 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6424 return PyBool_FromLong(Py_UNICODE_ISUPPER(*p) != 0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6425
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6426 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6427 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6428 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6429
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6430 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6431 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6432 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6433 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6434
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6435 if (Py_UNICODE_ISLOWER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6436 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6437 else if (!cased && Py_UNICODE_ISUPPER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6438 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6439 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6440 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6441 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6442
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6443 PyDoc_STRVAR(istitle__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6444 "S.istitle() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6445 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6446 Return True if S is a titlecased string and there is at least one\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6447 character in S, i.e. upper- and titlecase characters may only\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6448 follow uncased characters and lowercase characters only cased ones.\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6449 Return False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6450
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6451 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6452 unicode_istitle(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6453 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6454 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6455 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6456 int cased, previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6457
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6458 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6459 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6460 return PyBool_FromLong((Py_UNICODE_ISTITLE(*p) != 0) ||
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6461 (Py_UNICODE_ISUPPER(*p) != 0));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6462
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6463 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6464 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6465 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6466
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6467 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6468 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6469 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6470 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6471 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6472
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6473 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6474 if (previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6475 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6476 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6477 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6478 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6479 else if (Py_UNICODE_ISLOWER(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6480 if (!previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6481 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6482 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6483 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6484 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6485 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6486 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6487 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6488 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6489 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6490
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6491 PyDoc_STRVAR(isspace__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6492 "S.isspace() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6493 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6494 Return True if all characters in S are whitespace\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6495 and there is at least one character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6496
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6497 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6498 unicode_isspace(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6499 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6500 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6501 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6502
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6503 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6504 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6505 Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6506 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6507
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6508 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6509 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6510 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6511
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6512 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6513 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6514 if (!Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6515 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6516 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6517 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6518 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6519
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6520 PyDoc_STRVAR(isalpha__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6521 "S.isalpha() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6522 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6523 Return True if all characters in S are alphabetic\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6524 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6525
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6526 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6527 unicode_isalpha(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6528 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6529 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6530 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6531
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6532 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6533 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6534 Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6535 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6536
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6537 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6538 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6539 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6540
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6541 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6542 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6543 if (!Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6544 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6545 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6546 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6547 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6548
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6549 PyDoc_STRVAR(isalnum__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6550 "S.isalnum() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6551 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6552 Return True if all characters in S are alphanumeric\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6553 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6554
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6555 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6556 unicode_isalnum(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6557 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6558 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6559 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6560
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6561 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6562 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6563 Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6564 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6565
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6566 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6567 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6568 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6569
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6570 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6571 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6572 if (!Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6573 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6574 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6575 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6576 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
6577
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6578 PyDoc_STRVAR(isdecimal__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6579 "S.isdecimal() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6580 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6581 Return True if there are only decimal characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6582 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6583
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6584 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6585 unicode_isdecimal(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6586 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6587 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6588 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6589
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6590 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6591 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6592 Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6593 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6594
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6595 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6596 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6597 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6598
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6599 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6600 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6601 if (!Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6602 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6603 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6604 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6605 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6606
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6607 PyDoc_STRVAR(isdigit__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6608 "S.isdigit() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6609 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6610 Return True if all characters in S are digits\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
6611 and there is at least one character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6613 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6614 unicode_isdigit(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6615 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6616 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6617 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6618
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6619 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6620 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6621 Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6622 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6623
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6624 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6625 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6626 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6627
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6628 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6629 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6630 if (!Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6631 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6632 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6633 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6634 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6635
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6636 PyDoc_STRVAR(isnumeric__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6637 "S.isnumeric() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6638 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6639 Return True if there are only numeric characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6640 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6641
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6642 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6643 unicode_isnumeric(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6644 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6645 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6646 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6647
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6648 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6649 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6650 Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6651 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6652
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6653 /* Special case for empty strings */
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
6654 if (PyUnicode_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6655 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
6656
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6657 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6658 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6659 if (!Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6660 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6661 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6662 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6663 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6664
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6665 PyDoc_STRVAR(join__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6666 "S.join(sequence) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6667 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6668 Return a string which is the concatenation of the strings in the\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6669 sequence. The separator between elements is S.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6670
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6671 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6672 unicode_join(PyObject *self, PyObject *data)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6673 {
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6674 return PyUnicode_Join(self, data);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6675 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6676
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6677 static Py_ssize_t
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6678 unicode_length(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6679 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6680 return self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6681 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6682
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6683 PyDoc_STRVAR(ljust__doc__,
32197
3a84e23b042f - SF #962502: Add two more methods for unicode type; width() and
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31873
diff changeset
6684 "S.ljust(width[, fillchar]) -> int\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6685 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6686 Return S left justified in a Unicode string of length width. Padding is\n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6687 done using the specified fill character (default is a space).");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6688
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6689 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6690 unicode_ljust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6691 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6692 Py_ssize_t width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6693 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6694
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
6695 if (!PyArg_ParseTuple(args, "n|O&:ljust", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6696 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6697
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
6698 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6699 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6700 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6701 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6702
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6703 return (PyObject*) pad(self, 0, width - self->length, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6704 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6705
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6706 PyDoc_STRVAR(lower__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6707 "S.lower() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6708 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6709 Return a copy of the string S converted to lowercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6710
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6711 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6712 unicode_lower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6713 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6714 return fixup(self, fixlower);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6715 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6716
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6717 #define LEFTSTRIP 0
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6718 #define RIGHTSTRIP 1
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6719 #define BOTHSTRIP 2
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6720
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6721 /* Arrays indexed by above */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6722 static const char *stripformat[] = {"|O:lstrip", "|O:rstrip", "|O:strip"};
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6723
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6724 #define STRIPNAME(i) (stripformat[i]+3)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6725
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6726 /* externally visible for str.strip(unicode) */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6727 PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6728 _PyUnicode_XStrip(PyUnicodeObject *self, int striptype, PyObject *sepobj)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6729 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6730 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6731 Py_ssize_t len = PyUnicode_GET_SIZE(self);
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6732 Py_UNICODE *sep = PyUnicode_AS_UNICODE(sepobj);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6733 Py_ssize_t seplen = PyUnicode_GET_SIZE(sepobj);
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6734 Py_ssize_t i, j;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6735
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6736 BLOOM_MASK sepmask = make_bloom_mask(sep, seplen);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6737
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6738 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6739 if (striptype != RIGHTSTRIP) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6740 while (i < len && BLOOM_MEMBER(sepmask, s[i], sep, seplen)) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6741 i++;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6742 }
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6743 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6744
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6745 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6746 if (striptype != LEFTSTRIP) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6747 do {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6748 j--;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6749 } while (j >= i && BLOOM_MEMBER(sepmask, s[j], sep, seplen));
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6750 j++;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6751 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6752
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6753 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6754 Py_INCREF(self);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6755 return (PyObject*)self;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6756 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6757 else
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6758 return PyUnicode_FromUnicode(s+i, j-i);
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6759 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6760
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6761
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6762 static PyObject *
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6763 do_strip(PyUnicodeObject *self, int striptype)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6764 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6765 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6766 Py_ssize_t len = PyUnicode_GET_SIZE(self), i, j;
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6767
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6768 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6769 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6770 while (i < len && Py_UNICODE_ISSPACE(s[i])) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6771 i++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6772 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6773 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6774
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6775 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6776 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6777 do {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6778 j--;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6779 } while (j >= i && Py_UNICODE_ISSPACE(s[j]));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6780 j++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6781 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6782
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6783 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6784 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6785 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6786 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6787 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6788 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6789 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6790
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6791
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6792 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6793 do_argstrip(PyUnicodeObject *self, int striptype, PyObject *args)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6794 {
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6795 PyObject *sep = NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6796
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6797 if (!PyArg_ParseTuple(args, (char *)stripformat[striptype], &sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6798 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6799
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6800 if (sep != NULL && sep != Py_None) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6801 if (PyUnicode_Check(sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6802 return _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6803 else if (PyString_Check(sep)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6804 PyObject *res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6805 sep = PyUnicode_FromObject(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6806 if (sep==NULL)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6807 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6808 res = _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6809 Py_DECREF(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6810 return res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6811 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6812 else {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6813 PyErr_Format(PyExc_TypeError,
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6814 "%s arg must be None, unicode or str",
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6815 STRIPNAME(striptype));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6816 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6817 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6818 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6819
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6820 return do_strip(self, striptype);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6821 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6822
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6823
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6824 PyDoc_STRVAR(strip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6825 "S.strip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6826 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6827 Return a copy of the string S with leading and trailing\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6828 whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6829 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6830 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6831
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6832 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6833 unicode_strip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6834 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6835 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6836 return do_strip(self, BOTHSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6837 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6838 return do_argstrip(self, BOTHSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6839 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6840
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6841
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6842 PyDoc_STRVAR(lstrip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6843 "S.lstrip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6844 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6845 Return a copy of the string S with leading whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6846 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6847 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6848
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6849 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6850 unicode_lstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6851 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6852 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6853 return do_strip(self, LEFTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6854 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6855 return do_argstrip(self, LEFTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6856 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6857
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6858
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6859 PyDoc_STRVAR(rstrip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6860 "S.rstrip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6861 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6862 Return a copy of the string S with trailing whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6863 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
6864 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6865
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6866 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6867 unicode_rstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6868 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6869 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6870 return do_strip(self, RIGHTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6871 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6872 return do_argstrip(self, RIGHTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6873 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6874
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6875
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6876 static PyObject*
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6877 unicode_repeat(PyUnicodeObject *str, Py_ssize_t len)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6878 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6879 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6880 Py_UNICODE *p;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6881 Py_ssize_t nchars;
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6882 size_t nbytes;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6883
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6884 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6885 len = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6886
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
6887 if (len == 1 && PyUnicode_CheckExact(str)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6888 /* no repeat, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6889 Py_INCREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6890 return (PyObject*) str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6891 }
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6892
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6893 /* ensure # of chars needed doesn't overflow int and # of bytes
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6894 * needed doesn't overflow size_t
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6895 */
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6896 nchars = len * str->length;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6897 if (len && nchars / len != str->length) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6898 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6899 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6900 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6901 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6902 nbytes = (nchars + 1) * sizeof(Py_UNICODE);
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6903 if (nbytes / sizeof(Py_UNICODE) != (size_t)(nchars + 1)) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6904 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6905 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6906 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6907 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
6908 u = _PyUnicode_New(nchars);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6909 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6910 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6911
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6912 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6913
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6914 if (str->length == 1 && len > 0) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6915 Py_UNICODE_FILL(p, str->str[0], len);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6916 } else {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6917 Py_ssize_t done = 0; /* number of characters copied this far */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6918 if (done < nchars) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6919 Py_UNICODE_COPY(p, str->str, str->length);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6920 done = str->length;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6921 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6922 while (done < nchars) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6923 int n = (done <= nchars-done) ? done : nchars-done;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6924 Py_UNICODE_COPY(p+done, p, n);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6925 done += n;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
6926 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6927 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6928
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6929 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6930 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6931
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6932 PyObject *PyUnicode_Replace(PyObject *obj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6933 PyObject *subobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6934 PyObject *replobj,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6935 Py_ssize_t maxcount)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6936 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6937 PyObject *self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6938 PyObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6939 PyObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6940 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6941
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6942 self = PyUnicode_FromObject(obj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6943 if (self == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6944 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6945 str1 = PyUnicode_FromObject(subobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6946 if (str1 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6947 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6948 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6949 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6950 str2 = PyUnicode_FromObject(replobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6951 if (str2 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6952 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6953 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6954 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6955 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6956 result = replace((PyUnicodeObject *)self,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6957 (PyUnicodeObject *)str1,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6958 (PyUnicodeObject *)str2,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6959 maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6960 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6961 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6962 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6963 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6964 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6965
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6966 PyDoc_STRVAR(replace__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6967 "S.replace (old, new[, maxsplit]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6968 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6969 Return a copy of S with all occurrences of substring\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6970 old replaced by new. If the optional argument maxsplit is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6971 given, only the first maxsplit occurrences are replaced.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6972
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6973 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6974 unicode_replace(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6975 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6976 PyUnicodeObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6977 PyUnicodeObject *str2;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6978 Py_ssize_t maxcount = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6979 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6980
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
6981 if (!PyArg_ParseTuple(args, "OO|n:replace", &str1, &str2, &maxcount))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6982 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6983 str1 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6984 if (str1 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6985 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6986 str2 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str2);
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
6987 if (str2 == NULL) {
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
6988 Py_DECREF(str1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6989 return NULL;
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
6990 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6991
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6992 result = replace(self, str1, str2, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6993
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6994 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6995 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6996 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6997 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6998
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6999 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7000 PyObject *unicode_repr(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7001 {
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7002 PyObject *repr;
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7003 Py_UNICODE *p;
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7004 Py_UNICODE *s = PyUnicode_AS_UNICODE(unicode);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7005 Py_ssize_t size = PyUnicode_GET_SIZE(unicode);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7006
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7007 /* XXX(nnorwitz): rather than over-allocating, it would be
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7008 better to choose a different scheme. Perhaps scan the
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7009 first N-chars of the string and allocate based on that size.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7010 */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7011 /* Initial allocation is based on the longest-possible unichr
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7012 escape.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7013
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7014 In wide (UTF-32) builds '\U00xxxxxx' is 10 chars per source
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7015 unichr, so in this case it's the longest unichr escape. In
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7016 narrow (UTF-16) builds this is five chars per source unichr
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7017 since there are two unichrs in the surrogate pair, so in narrow
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7018 (UTF-16) builds it's not the longest unichr escape.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7019
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7020 In wide or narrow builds '\uxxxx' is 6 chars per source unichr,
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7021 so in the narrow (UTF-16) build case it's the longest unichr
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7022 escape.
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7023 */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7024
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7025 repr = PyUnicode_FromUnicode(NULL,
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7026 2 /* quotes */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7027 #ifdef Py_UNICODE_WIDE
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7028 + 10*size
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7029 #else
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7030 + 6*size
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7031 #endif
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7032 + 1);
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7033 if (repr == NULL)
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7034 return NULL;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7035
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7036 p = PyUnicode_AS_UNICODE(repr);
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7037
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7038 /* Add quote */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7039 *p++ = (findchar(s, size, '\'') &&
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7040 !findchar(s, size, '"')) ? '"' : '\'';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7041 while (size-- > 0) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7042 Py_UNICODE ch = *s++;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7043
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7044 /* Escape quotes and backslashes */
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7045 if ((ch == PyUnicode_AS_UNICODE(repr)[0]) || (ch == '\\')) {
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7046 *p++ = '\\';
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7047 *p++ = ch;
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7048 continue;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7049 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7050
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7051 #ifdef Py_UNICODE_WIDE
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7052 /* Map 21-bit characters to '\U00xxxxxx' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7053 else if (ch >= 0x10000) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7054 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7055 *p++ = 'U';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7056 *p++ = hexdigits[(ch >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7057 *p++ = hexdigits[(ch >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7058 *p++ = hexdigits[(ch >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7059 *p++ = hexdigits[(ch >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7060 *p++ = hexdigits[(ch >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7061 *p++ = hexdigits[(ch >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7062 *p++ = hexdigits[(ch >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7063 *p++ = hexdigits[ch & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7064 continue;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7065 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7066 #else
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7067 /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7068 else if (ch >= 0xD800 && ch < 0xDC00) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7069 Py_UNICODE ch2;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7070 Py_UCS4 ucs;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7071
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7072 ch2 = *s++;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7073 size--;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7074 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7075 ucs = (((ch & 0x03FF) << 10) | (ch2 & 0x03FF)) + 0x00010000;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7076 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7077 *p++ = 'U';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7078 *p++ = hexdigits[(ucs >> 28) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7079 *p++ = hexdigits[(ucs >> 24) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7080 *p++ = hexdigits[(ucs >> 20) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7081 *p++ = hexdigits[(ucs >> 16) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7082 *p++ = hexdigits[(ucs >> 12) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7083 *p++ = hexdigits[(ucs >> 8) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7084 *p++ = hexdigits[(ucs >> 4) & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7085 *p++ = hexdigits[ucs & 0x0000000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7086 continue;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7087 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7088 /* Fall through: isolated surrogates are copied as-is */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7089 s--;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7090 size++;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7091 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7092 #endif
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7093
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7094 /* Map 16-bit characters to '\uxxxx' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7095 if (ch >= 256) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7096 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7097 *p++ = 'u';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7098 *p++ = hexdigits[(ch >> 12) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7099 *p++ = hexdigits[(ch >> 8) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7100 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7101 *p++ = hexdigits[ch & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7102 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7103
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7104 /* Map special whitespace to '\t', \n', '\r' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7105 else if (ch == '\t') {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7106 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7107 *p++ = 't';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7108 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7109 else if (ch == '\n') {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7110 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7111 *p++ = 'n';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7112 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7113 else if (ch == '\r') {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7114 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7115 *p++ = 'r';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7116 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7117
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7118 /* Map non-printable US ASCII to '\xhh' */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7119 else if (ch < ' ' || ch >= 0x7F) {
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7120 *p++ = '\\';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7121 *p++ = 'x';
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7122 *p++ = hexdigits[(ch >> 4) & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7123 *p++ = hexdigits[ch & 0x000F];
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7124 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7125
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7126 /* Copy everything else as-is */
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7127 else
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7128 *p++ = (char) ch;
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7129 }
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7130 /* Add quote */
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7131 *p++ = PyUnicode_AS_UNICODE(repr)[0];
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7132
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7133 *p = '\0';
41769
b29d71d43c80 Add functions PyUnicode_Append() and PyUnicode_AppendAndDel() that mirror
Walter Dörwald <walter@livinglogic.de>
parents: 41766
diff changeset
7134 _PyUnicode_Resize(&repr, p - PyUnicode_AS_UNICODE(repr));
41703
7993f23ad46c Change PyUnicode_EncodeUnicodeEscape() to return a bytes object.
Walter Dörwald <walter@livinglogic.de>
parents: 41658
diff changeset
7135 return repr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7136 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7137
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7138 PyDoc_STRVAR(rfind__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7139 "S.rfind(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7140 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7141 Return the highest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7142 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7143 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7144 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7145 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7146
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7147 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7148 unicode_rfind(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7149 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7150 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7151 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7152 Py_ssize_t end = PY_SSIZE_T_MAX;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7153 Py_ssize_t result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7154
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7155 if (!PyArg_ParseTuple(args, "O|O&O&:rfind", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7156 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7157 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7158 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7159 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7160 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7161
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7162 result = stringlib_rfind_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7163 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7164 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7165 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7166 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7167
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7168 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7169
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7170 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7171 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7172
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7173 PyDoc_STRVAR(rindex__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7174 "S.rindex(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7175 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7176 Like S.rfind() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7177
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7178 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7179 unicode_rindex(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7180 {
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7181 PyObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7182 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7183 Py_ssize_t end = PY_SSIZE_T_MAX;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7184 Py_ssize_t result;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7185
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7186 if (!PyArg_ParseTuple(args, "O|O&O&:rindex", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7187 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7188 return NULL;
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7189 substring = PyUnicode_FromObject(substring);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7190 if (!substring)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7191 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7192
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7193 result = stringlib_rfind_slice(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7194 PyUnicode_AS_UNICODE(self), PyUnicode_GET_SIZE(self),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7195 PyUnicode_AS_UNICODE(substring), PyUnicode_GET_SIZE(substring),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7196 start, end
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7197 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7198
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7199 Py_DECREF(substring);
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7200
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7201 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7202 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7203 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7204 }
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7205 return PyInt_FromSsize_t(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7206 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7207
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7208 PyDoc_STRVAR(rjust__doc__,
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7209 "S.rjust(width[, fillchar]) -> unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7210 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7211 Return S right justified in a Unicode string of length width. Padding is\n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7212 done using the specified fill character (default is a space).");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7213
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7214 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7215 unicode_rjust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7216 {
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7217 Py_ssize_t width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7218 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7219
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7220 if (!PyArg_ParseTuple(args, "n|O&:rjust", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7221 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7222
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
7223 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7224 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7225 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7226 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7227
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
7228 return (PyObject*) pad(self, width - self->length, 0, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7229 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7230
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7231 static PyObject*
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7232 unicode_slice(PyUnicodeObject *self, Py_ssize_t start, Py_ssize_t end)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7233 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7234 /* standard clamping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7235 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7236 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7237 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7238 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7239 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7240 end = self->length;
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
7241 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7242 /* full slice, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7243 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7244 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7245 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7246 if (start > end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7247 start = end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7248 /* copy slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7249 return (PyObject*) PyUnicode_FromUnicode(self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7250 end - start);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7251 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7252
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7253 PyObject *PyUnicode_Split(PyObject *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7254 PyObject *sep,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7255 Py_ssize_t maxsplit)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7256 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7257 PyObject *result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7258
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7259 s = PyUnicode_FromObject(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7260 if (s == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7261 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7262 if (sep != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7263 sep = PyUnicode_FromObject(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7264 if (sep == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7265 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7266 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7267 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7268 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7269
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7270 result = split((PyUnicodeObject *)s, (PyUnicodeObject *)sep, maxsplit);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7271
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7272 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7273 Py_XDECREF(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7274 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7275 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7276
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7277 PyDoc_STRVAR(split__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7278 "S.split([sep [,maxsplit]]) -> list of strings\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7279 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7280 Return a list of the words in S, using sep as the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7281 delimiter string. If maxsplit is given, at most maxsplit\n\
33495
2d6e1854082e Add a missing line continuation character.
Thomas Heller <theller@ctypes.org>
parents: 33487
diff changeset
7282 splits are done. If sep is not specified or is None,\n\
33486
53c1357c9c63 Enhance the docstrings for unicode.split() and string.split()
Walter Dörwald <walter@livinglogic.de>
parents: 33428
diff changeset
7283 any whitespace string is a separator.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7284
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7285 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7286 unicode_split(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7287 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7288 PyObject *substring = Py_None;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7289 Py_ssize_t maxcount = -1;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7290
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7291 if (!PyArg_ParseTuple(args, "|On:split", &substring, &maxcount))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7292 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7293
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7294 if (substring == Py_None)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7295 return split(self, NULL, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7296 else if (PyUnicode_Check(substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7297 return split(self, (PyUnicodeObject *)substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7298 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7299 return PyUnicode_Split((PyObject *)self, substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7300 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7301
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7302 PyObject *
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7303 PyUnicode_Partition(PyObject *str_in, PyObject *sep_in)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7304 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7305 PyObject* str_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7306 PyObject* sep_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7307 PyObject* out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7308
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7309 str_obj = PyUnicode_FromObject(str_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7310 if (!str_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7311 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7312 sep_obj = PyUnicode_FromObject(sep_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7313 if (!sep_obj) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7314 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7315 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7316 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7317
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7318 out = stringlib_partition(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7319 str_obj, PyUnicode_AS_UNICODE(str_obj), PyUnicode_GET_SIZE(str_obj),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7320 sep_obj, PyUnicode_AS_UNICODE(sep_obj), PyUnicode_GET_SIZE(sep_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7321 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7322
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7323 Py_DECREF(sep_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7324 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7325
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7326 return out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7327 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7328
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7329
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7330 PyObject *
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7331 PyUnicode_RPartition(PyObject *str_in, PyObject *sep_in)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7332 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7333 PyObject* str_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7334 PyObject* sep_obj;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7335 PyObject* out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7336
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7337 str_obj = PyUnicode_FromObject(str_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7338 if (!str_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7339 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7340 sep_obj = PyUnicode_FromObject(sep_in);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7341 if (!sep_obj) {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7342 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7343 return NULL;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7344 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7345
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7346 out = stringlib_rpartition(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7347 str_obj, PyUnicode_AS_UNICODE(str_obj), PyUnicode_GET_SIZE(str_obj),
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7348 sep_obj, PyUnicode_AS_UNICODE(sep_obj), PyUnicode_GET_SIZE(sep_obj)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7349 );
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7350
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7351 Py_DECREF(sep_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7352 Py_DECREF(str_obj);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7353
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7354 return out;
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7355 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7356
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7357 PyDoc_STRVAR(partition__doc__,
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7358 "S.partition(sep) -> (head, sep, tail)\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7359 \n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7360 Searches for the separator sep in S, and returns the part before it,\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7361 the separator itself, and the part after it. If the separator is not\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7362 found, returns S and two empty strings.");
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7363
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7364 static PyObject*
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7365 unicode_partition(PyUnicodeObject *self, PyObject *separator)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7366 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7367 return PyUnicode_Partition((PyObject *)self, separator);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7368 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7369
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7370 PyDoc_STRVAR(rpartition__doc__,
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
7371 "S.rpartition(sep) -> (tail, sep, head)\n\
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7372 \n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7373 Searches for the separator sep in S, starting at the end of S, and returns\n\
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7374 the part before it, the separator itself, and the part after it. If the\n\
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
7375 separator is not found, returns two empty strings and S.");
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7376
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7377 static PyObject*
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7378 unicode_rpartition(PyUnicodeObject *self, PyObject *separator)
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7379 {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7380 return PyUnicode_RPartition((PyObject *)self, separator);
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7381 }
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7382
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7383 PyObject *PyUnicode_RSplit(PyObject *s,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7384 PyObject *sep,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7385 Py_ssize_t maxsplit)
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7386 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7387 PyObject *result;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7388
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7389 s = PyUnicode_FromObject(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7390 if (s == NULL)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7391 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7392 if (sep != NULL) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7393 sep = PyUnicode_FromObject(sep);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7394 if (sep == NULL) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7395 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7396 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7397 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7398 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7399
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7400 result = rsplit((PyUnicodeObject *)s, (PyUnicodeObject *)sep, maxsplit);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7401
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7402 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7403 Py_XDECREF(sep);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7404 return result;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7405 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7406
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7407 PyDoc_STRVAR(rsplit__doc__,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7408 "S.rsplit([sep [,maxsplit]]) -> list of strings\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7409 \n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7410 Return a list of the words in S, using sep as the\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7411 delimiter string, starting at the end of the string and\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7412 working to the front. If maxsplit is given, at most maxsplit\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7413 splits are done. If sep is not specified, any whitespace string\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7414 is a separator.");
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7415
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7416 static PyObject*
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7417 unicode_rsplit(PyUnicodeObject *self, PyObject *args)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7418 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7419 PyObject *substring = Py_None;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7420 Py_ssize_t maxcount = -1;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7421
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7422 if (!PyArg_ParseTuple(args, "|On:rsplit", &substring, &maxcount))
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7423 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7424
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7425 if (substring == Py_None)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7426 return rsplit(self, NULL, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7427 else if (PyUnicode_Check(substring))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7428 return rsplit(self, (PyUnicodeObject *)substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7429 else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7430 return PyUnicode_RSplit((PyObject *)self, substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7431 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7432
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7433 PyDoc_STRVAR(splitlines__doc__,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7434 "S.splitlines([keepends]]) -> list of strings\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7435 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7436 Return a list of the lines in S, breaking at line boundaries.\n\
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7437 Line breaks are not included in the resulting list unless keepends\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7438 is given and true.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7439
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7440 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7441 unicode_splitlines(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7442 {
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7443 int keepends = 0;
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7444
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7445 if (!PyArg_ParseTuple(args, "|i:splitlines", &keepends))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7446 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7447
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
7448 return PyUnicode_Splitlines((PyObject *)self, keepends);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7449 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7450
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7451 static
41595
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
7452 PyObject *unicode_str(PyObject *self)
2c54763c4504 More coding by random modification.
Guido van Rossum <guido@python.org>
parents: 41585
diff changeset
7453 {
41919
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7454 if (PyUnicode_CheckExact(self)) {
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7455 Py_INCREF(self);
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7456 return self;
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7457 } else
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7458 /* Subtype -- return genuine unicode string with the same value. */
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7459 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(self),
dd4689d0ec85 Add support for width, precision and zeropadding to the %d, %i, %u and %x
Walter Dörwald <walter@livinglogic.de>
parents: 41876
diff changeset
7460 PyUnicode_GET_SIZE(self));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7461 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7462
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7463 PyDoc_STRVAR(swapcase__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7464 "S.swapcase() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7465 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7466 Return a copy of S with uppercase characters converted to lowercase\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7467 and vice versa.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7468
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7469 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7470 unicode_swapcase(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7471 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7472 return fixup(self, fixswapcase);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7473 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7474
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7475 PyDoc_STRVAR(translate__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7476 "S.translate(table) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7477 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7478 Return a copy of the string S, where all characters have been mapped\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7479 through the given translation table, which must be a mapping of\n\
25120
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
7480 Unicode ordinals to Unicode ordinals, Unicode strings or None.\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
7481 Unmapped characters are left untouched. Characters mapped to None\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
7482 are deleted.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7483
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7484 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7485 unicode_translate(PyUnicodeObject *self, PyObject *table)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7486 {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7487 return PyUnicode_TranslateCharmap(self->str,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7488 self->length,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7489 table,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7490 "ignore");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7491 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7492
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7493 PyDoc_STRVAR(upper__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7494 "S.upper() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7495 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7496 Return a copy of S converted to uppercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7497
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7498 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7499 unicode_upper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7500 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7501 return fixup(self, fixupper);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7502 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7503
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7504 PyDoc_STRVAR(zfill__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7505 "S.zfill(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7506 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7507 Pad a numeric string x with zeros on the left, to fill a field\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7508 of the specified width. The string x is never truncated.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7509
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7510 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7511 unicode_zfill(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7512 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7513 Py_ssize_t fill;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7514 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7515
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7516 Py_ssize_t width;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7517 if (!PyArg_ParseTuple(args, "n:zfill", &width))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7518 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7519
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7520 if (self->length >= width) {
23051
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7521 if (PyUnicode_CheckExact(self)) {
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7522 Py_INCREF(self);
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7523 return (PyObject*) self;
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7524 }
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7525 else
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7526 return PyUnicode_FromUnicode(
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7527 PyUnicode_AS_UNICODE(self),
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7528 PyUnicode_GET_SIZE(self)
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
7529 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7530 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7531
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7532 fill = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7533
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7534 u = pad(self, fill, 0, '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7535
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7536 if (u == NULL)
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7537 return NULL;
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7538
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7539 if (u->str[fill] == '+' || u->str[fill] == '-') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7540 /* move sign to beginning of string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7541 u->str[0] = u->str[fill];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7542 u->str[fill] = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7543 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7544
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7545 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7546 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7547
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7548 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7549 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7550 unicode_freelistsize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7551 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7552 return PyInt_FromLong(unicode_freelist_size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7553 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7554 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7555
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7556 PyDoc_STRVAR(startswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
7557 "S.startswith(prefix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7558 \n\
28559
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7559 Return True if S starts with the specified prefix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7560 With optional start, test S beginning at that position.\n\
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7561 With optional end, stop comparing S at that position.\n\
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7562 prefix can also be a tuple of strings to try.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7563
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7564 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7565 unicode_startswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7566 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7567 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7568 PyObject *subobj;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7569 PyUnicodeObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7570 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7571 Py_ssize_t end = PY_SSIZE_T_MAX;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7572 int result;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7573
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7574 if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj,
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
7575 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7576 return NULL;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7577 if (PyTuple_Check(subobj)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7578 Py_ssize_t i;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7579 for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7580 substring = (PyUnicodeObject *)PyUnicode_FromObject(
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7581 PyTuple_GET_ITEM(subobj, i));
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7582 if (substring == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7583 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7584 result = tailmatch(self, substring, start, end, -1);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7585 Py_DECREF(substring);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7586 if (result) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7587 Py_RETURN_TRUE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7588 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7589 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7590 /* nothing matched */
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7591 Py_RETURN_FALSE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7592 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7593 substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7594 if (substring == NULL)
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7595 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7596 result = tailmatch(self, substring, start, end, -1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7597 Py_DECREF(substring);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7598 return PyBool_FromLong(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7599 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7600
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7601
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7602 PyDoc_STRVAR(endswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
7603 "S.endswith(suffix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7604 \n\
28559
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7605 Return True if S ends with the specified suffix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
7606 With optional start, test S beginning at that position.\n\
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7607 With optional end, stop comparing S at that position.\n\
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7608 suffix can also be a tuple of strings to try.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7609
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7610 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7611 unicode_endswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7612 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7613 {
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7614 PyObject *subobj;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7615 PyUnicodeObject *substring;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7616 Py_ssize_t start = 0;
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7617 Py_ssize_t end = PY_SSIZE_T_MAX;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7618 int result;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7619
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7620 if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj,
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7621 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7622 return NULL;
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7623 if (PyTuple_Check(subobj)) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7624 Py_ssize_t i;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7625 for (i = 0; i < PyTuple_GET_SIZE(subobj); i++) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7626 substring = (PyUnicodeObject *)PyUnicode_FromObject(
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7627 PyTuple_GET_ITEM(subobj, i));
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7628 if (substring == NULL)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7629 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7630 result = tailmatch(self, substring, start, end, +1);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7631 Py_DECREF(substring);
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7632 if (result) {
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7633 Py_RETURN_TRUE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7634 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7635 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7636 Py_RETURN_FALSE;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7637 }
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7638 substring = (PyUnicodeObject *)PyUnicode_FromObject(subobj);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7639 if (substring == NULL)
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7640 return NULL;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7641
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7642 result = tailmatch(self, substring, start, end, +1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7643 Py_DECREF(substring);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
7644 return PyBool_FromLong(result);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7645 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7646
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7647
27521
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7648
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7649 static PyObject *
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7650 unicode_getnewargs(PyUnicodeObject *v)
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7651 {
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7652 return Py_BuildValue("(u#)", v->str, v->length);
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7653 }
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7654
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7655
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7656 static PyMethodDef unicode_methods[] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7657
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7658 /* Order is according to common usage: often used methods should
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7659 appear first, since lookup is done sequentially. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7660
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7661 {"encode", (PyCFunction) unicode_encode, METH_VARARGS, encode__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7662 {"replace", (PyCFunction) unicode_replace, METH_VARARGS, replace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7663 {"split", (PyCFunction) unicode_split, METH_VARARGS, split__doc__},
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
7664 {"rsplit", (PyCFunction) unicode_rsplit, METH_VARARGS, rsplit__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7665 {"join", (PyCFunction) unicode_join, METH_O, join__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7666 {"capitalize", (PyCFunction) unicode_capitalize, METH_NOARGS, capitalize__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7667 {"title", (PyCFunction) unicode_title, METH_NOARGS, title__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7668 {"center", (PyCFunction) unicode_center, METH_VARARGS, center__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7669 {"count", (PyCFunction) unicode_count, METH_VARARGS, count__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7670 {"expandtabs", (PyCFunction) unicode_expandtabs, METH_VARARGS, expandtabs__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7671 {"find", (PyCFunction) unicode_find, METH_VARARGS, find__doc__},
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7672 {"partition", (PyCFunction) unicode_partition, METH_O, partition__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7673 {"index", (PyCFunction) unicode_index, METH_VARARGS, index__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7674 {"ljust", (PyCFunction) unicode_ljust, METH_VARARGS, ljust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7675 {"lower", (PyCFunction) unicode_lower, METH_NOARGS, lower__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
7676 {"lstrip", (PyCFunction) unicode_lstrip, METH_VARARGS, lstrip__doc__},
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
7677 {"decode", (PyCFunction) unicode_decode, METH_VARARGS, decode__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7678 /* {"maketrans", (PyCFunction) unicode_maketrans, METH_VARARGS, maketrans__doc__}, */
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7679 {"rfind", (PyCFunction) unicode_rfind, METH_VARARGS, rfind__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7680 {"rindex", (PyCFunction) unicode_rindex, METH_VARARGS, rindex__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7681 {"rjust", (PyCFunction) unicode_rjust, METH_VARARGS, rjust__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
7682 {"rstrip", (PyCFunction) unicode_rstrip, METH_VARARGS, rstrip__doc__},
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
7683 {"rpartition", (PyCFunction) unicode_rpartition, METH_O, rpartition__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7684 {"splitlines", (PyCFunction) unicode_splitlines, METH_VARARGS, splitlines__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
7685 {"strip", (PyCFunction) unicode_strip, METH_VARARGS, strip__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7686 {"swapcase", (PyCFunction) unicode_swapcase, METH_NOARGS, swapcase__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7687 {"translate", (PyCFunction) unicode_translate, METH_O, translate__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7688 {"upper", (PyCFunction) unicode_upper, METH_NOARGS, upper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7689 {"startswith", (PyCFunction) unicode_startswith, METH_VARARGS, startswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7690 {"endswith", (PyCFunction) unicode_endswith, METH_VARARGS, endswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7691 {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7692 {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7693 {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7694 {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7695 {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7696 {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7697 {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7698 {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7699 {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
7700 {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7701 #if 0
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7702 {"capwords", (PyCFunction) unicode_capwords, METH_NOARGS, capwords__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7703 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7704
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7705 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7706 /* This one is just used for debugging the implementation. */
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
7707 {"freelistsize", (PyCFunction) unicode_freelistsize, METH_NOARGS},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7708 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7709
27521
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
7710 {"__getnewargs__", (PyCFunction)unicode_getnewargs, METH_NOARGS},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7711 {NULL, NULL}
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7712 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7713
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7714 static PyObject *
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7715 unicode_mod(PyObject *v, PyObject *w)
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7716 {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7717 if (!PyUnicode_Check(v)) {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7718 Py_INCREF(Py_NotImplemented);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7719 return Py_NotImplemented;
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7720 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7721 return PyUnicode_Format(v, w);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7722 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7723
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7724 static PyNumberMethods unicode_as_number = {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7725 0, /*nb_add*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7726 0, /*nb_subtract*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7727 0, /*nb_multiply*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7728 unicode_mod, /*nb_remainder*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7729 };
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7730
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7731 static PySequenceMethods unicode_as_sequence = {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7732 (lenfunc) unicode_length, /* sq_length */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7733 PyUnicode_Concat, /* sq_concat */
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7734 (ssizeargfunc) unicode_repeat, /* sq_repeat */
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7735 (ssizeargfunc) unicode_getitem, /* sq_item */
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7736 (ssizessizeargfunc) unicode_slice, /* sq_slice */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7737 0, /* sq_ass_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7738 0, /* sq_ass_slice */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7739 PyUnicode_Contains, /* sq_contains */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7740 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7741
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7742 static PyObject*
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7743 unicode_subscript(PyUnicodeObject* self, PyObject* item)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7744 {
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
7745 if (PyIndex_Check(item)) {
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
7746 Py_ssize_t i = PyNumber_AsSsize_t(item, PyExc_IndexError);
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7747 if (i == -1 && PyErr_Occurred())
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7748 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7749 if (i < 0)
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7750 i += PyUnicode_GET_SIZE(self);
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7751 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7752 } else if (PySlice_Check(item)) {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7753 Py_ssize_t start, stop, step, slicelength, cur, i;
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7754 Py_UNICODE* source_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7755 Py_UNICODE* result_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7756 PyObject* result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7757
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7758 if (PySlice_GetIndicesEx((PySliceObject*)item, PyUnicode_GET_SIZE(self),
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7759 &start, &stop, &step, &slicelength) < 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7760 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7761 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7762
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7763 if (slicelength <= 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7764 return PyUnicode_FromUnicode(NULL, 0);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7765 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7766 source_buf = PyUnicode_AS_UNICODE((PyObject*)self);
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7767 result_buf = (Py_UNICODE *)PyMem_MALLOC(slicelength*
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
7768 sizeof(Py_UNICODE));
36203
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7769
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7770 if (result_buf == NULL)
44cfca6136c6 Stop maintaining the buildno file.
Martin v. Löwis <martin@v.loewis.de>
parents: 36056
diff changeset
7771 return PyErr_NoMemory();
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7772
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7773 for (cur = start, i = 0; i < slicelength; cur += step, i++) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7774 result_buf[i] = source_buf[cur];
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7775 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7776
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7777 result = PyUnicode_FromUnicode(result_buf, slicelength);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7778 PyMem_FREE(result_buf);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7779 return result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7780 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7781 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7782 PyErr_SetString(PyExc_TypeError, "string indices must be integers");
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7783 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7784 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7785 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7786
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7787 static PyMappingMethods unicode_as_mapping = {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7788 (lenfunc)unicode_length, /* mp_length */
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7789 (binaryfunc)unicode_subscript, /* mp_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7790 (objobjargproc)0, /* mp_ass_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7791 };
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7792
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7793 static Py_ssize_t
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7794 unicode_buffer_getreadbuf(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7795 Py_ssize_t index,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7796 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7797 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7798 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7799 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7800 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7801 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7802 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7803 *ptr = (void *) self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7804 return PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7805 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7806
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7807 static Py_ssize_t
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7808 unicode_buffer_getwritebuf(PyUnicodeObject *self, Py_ssize_t index,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7809 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7810 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7811 PyErr_SetString(PyExc_TypeError,
23837
359a57e96d5e Fix typo in exception message
Neal Norwitz <nnorwitz@gmail.com>
parents: 23832
diff changeset
7812 "cannot use unicode as modifiable buffer");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7813 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7814 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7815
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7816 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7817 unicode_buffer_getsegcount(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7818 Py_ssize_t *lenp)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7819 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7820 if (lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7821 *lenp = PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7822 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7823 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7824
36519
d6d415df0df0 Use Py_ssize_t for counts and sizes.
Martin v. Löwis <martin@v.loewis.de>
parents: 36510
diff changeset
7825 static Py_ssize_t
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7826 unicode_buffer_getcharbuf(PyUnicodeObject *self,
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7827 Py_ssize_t index,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7828 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7829 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7830 PyObject *str;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7831
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7832 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7833 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7834 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7835 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7836 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
7837 str = _PyUnicode_AsDefaultEncodedString((PyObject *)self, NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7838 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7839 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7840 *ptr = (void *) PyString_AS_STRING(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7841 return PyString_GET_SIZE(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7842 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7843
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7844 /* Helpers for PyUnicode_Format() */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7845
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7846 static PyObject *
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7847 getnextarg(PyObject *args, Py_ssize_t arglen, Py_ssize_t *p_argidx)
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7848 {
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7849 Py_ssize_t argidx = *p_argidx;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7850 if (argidx < arglen) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7851 (*p_argidx)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7852 if (arglen < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7853 return args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7854 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7855 return PyTuple_GetItem(args, argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7856 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7857 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7858 "not enough arguments for format string");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7859 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7860 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7861
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7862 #define F_LJUST (1<<0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7863 #define F_SIGN (1<<1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7864 #define F_BLANK (1<<2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7865 #define F_ALT (1<<3)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7866 #define F_ZERO (1<<4)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7867
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7868 static Py_ssize_t
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7869 strtounicode(Py_UNICODE *buffer, const char *charbuffer)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7870 {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7871 register Py_ssize_t i;
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
7872 Py_ssize_t len = strlen(charbuffer);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7873 for (i = len - 1; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7874 buffer[i] = (Py_UNICODE) charbuffer[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7875
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7876 return len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7877 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7878
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7879 static int
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7880 doubletounicode(Py_UNICODE *buffer, size_t len, const char *format, double x)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7881 {
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7882 Py_ssize_t result;
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7883
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7884 PyOS_ascii_formatd((char *)buffer, len, format, x);
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7885 result = strtounicode(buffer, (char *)buffer);
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7886 return Py_SAFE_DOWNCAST(result, Py_ssize_t, int);
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7887 }
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7888
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7889 static int
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7890 longtounicode(Py_UNICODE *buffer, size_t len, const char *format, long x)
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7891 {
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7892 Py_ssize_t result;
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7893
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7894 PyOS_snprintf((char *)buffer, len, format, x);
36508
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7895 result = strtounicode(buffer, (char *)buffer);
7322dccba224 doubletounicode(), longtounicode():
Tim Peters <tim.peters@gmail.com>
parents: 36506
diff changeset
7896 return Py_SAFE_DOWNCAST(result, Py_ssize_t, int);
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7897 }
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7898
24736
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7899 /* XXX To save some code duplication, formatfloat/long/int could have been
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7900 shared with stringobject.c, converting from 8-bit to Unicode after the
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7901 formatting is done. */
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
7902
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7903 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7904 formatfloat(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7905 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7906 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7907 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7908 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7909 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7910 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7911 /* fmt = '%#.' + `prec` + `type`
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7912 worst case length = 3 + 10 (len of INT_MAX) + 1 = 14 (use 20)*/
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7913 char fmt[20];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7914 double x;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7915
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7916 x = PyFloat_AsDouble(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7917 if (x == -1.0 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7918 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7919 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7920 prec = 6;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7921 if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7922 type = 'g';
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7923 /* Worst case length calc to ensure no buffer overrun:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7924
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7925 'g' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7926 fmt = %#.<prec>g
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7927 buf = '-' + [0-9]*prec + '.' + 'e+' + (longest exp
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7928 for any double rep.)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7929 len = 1 + prec + 1 + 2 + 5 = 9 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7930
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7931 'f' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7932 buf = '-' + [0-9]*x + '.' + [0-9]*prec (with x < 50)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7933 len = 1 + 50 + 1 + prec = 52 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7934
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7935 If prec=0 the effective precision is 1 (the leading digit is
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7936 always given), therefore increase the length by one.
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7937
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7938 */
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7939 if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) ||
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7940 (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) {
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7941 PyErr_SetString(PyExc_OverflowError,
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7942 "formatted float is too long (precision too large?)");
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7943 return -1;
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7944 }
21082
c8d775b9ff92 formatfloat(), formatint(): Conversion of sprintf() to PyOS_snprintf()
Barry Warsaw <barry@python.org>
parents: 21067
diff changeset
7945 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%d%c",
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7946 (flags&F_ALT) ? "#" : "",
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
7947 prec, type);
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
7948 return doubletounicode(buf, buflen, fmt, x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7949 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7950
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7951 static PyObject*
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7952 formatlong(PyObject *val, int flags, int prec, int type)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7953 {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7954 char *buf;
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7955 int len;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7956 PyObject *str; /* temporary string object. */
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7957 PyObject *result;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7958
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7959 str = _PyString_FormatLong(val, flags, prec, type, &buf, &len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7960 if (!str)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7961 return NULL;
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7962 result = PyUnicode_FromStringAndSize(buf, len);
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7963 Py_DECREF(str);
42123
383c8907bea0 Silence GCC warning about uninitialzed variable.
Walter Dörwald <walter@livinglogic.de>
parents: 42084
diff changeset
7964 return result;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7965 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7966
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7967 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7968 formatint(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7969 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7970 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7971 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7972 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7973 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7974 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7975 /* fmt = '%#.' + `prec` + 'l' + `type`
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7976 * worst case length = 3 + 19 (worst len of INT_MAX on 64-bit machine)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7977 * + 1 + 1
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7978 * = 24
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7979 */
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7980 char fmt[64]; /* plenty big enough! */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7981 char *sign;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7982 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7983
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7984 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7985 if (x == -1 && PyErr_Occurred())
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7986 return -1;
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7987 if (x < 0 && type == 'u') {
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7988 type = 'd';
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7989 }
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7990 if (x < 0 && (type == 'x' || type == 'X' || type == 'o'))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7991 sign = "-";
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7992 else
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7993 sign = "";
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7994 if (prec < 0)
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7995 prec = 1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
7996
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7997 /* buf = '+'/'-'/'' + '0'/'0x'/'' + '[0-9]'*max(prec, len(x in octal))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7998 * worst case buf = '-0x' + [0-9]*prec, where prec >= 11
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7999 */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8000 if (buflen <= 14 || buflen <= (size_t)3 + (size_t)prec) {
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8001 PyErr_SetString(PyExc_OverflowError,
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8002 "formatted integer is too long (precision too large?)");
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8003 return -1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8004 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8005
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8006 if ((flags & F_ALT) &&
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8007 (type == 'x' || type == 'X' || type == 'o')) {
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8008 /* When converting under %#o, %#x or %#X, there are a number
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8009 * of issues that cause pain:
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8010 * - for %#o, we want a different base marker than C
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8011 * - when 0 is being converted, the C standard leaves off
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8012 * the '0x' or '0X', which is inconsistent with other
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8013 * %#x/%#X conversions and inconsistent with Python's
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8014 * hex() function
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8015 * - there are platforms that violate the standard and
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8016 * convert 0 with the '0x' or '0X'
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8017 * (Metrowerks, Compaq Tru64)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8018 * - there are platforms that give '0x' when converting
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8019 * under %#X, but convert 0 in accordance with the
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8020 * standard (OS/2 EMX)
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8021 *
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8022 * We can achieve the desired consistency by inserting our
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8023 * own '0x' or '0X' prefix, and substituting %x/%X in place
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8024 * of %#x/%#X.
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8025 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8026 * Note that this is the same approach as used in
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8027 * formatint() in stringobject.c
22053
49197fd76362 OS/2 EMX port changes (Objects part of patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 21886
diff changeset
8028 */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8029 PyOS_snprintf(fmt, sizeof(fmt), "%s0%c%%.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8030 sign, type, prec, type);
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8031 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8032 else {
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8033 PyOS_snprintf(fmt, sizeof(fmt), "%s%%%s.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8034 sign, (flags&F_ALT) ? "#" : "",
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8035 prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
8036 }
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8037 if (sign[0])
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
8038 return longtounicode(buf, buflen, fmt, -x);
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8039 else
36261
303dea847639 - Patch #1400181, fix unicode string formatting to not use the locale.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36236
diff changeset
8040 return longtounicode(buf, buflen, fmt, x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8041 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8042
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8043 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8044 formatchar(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8045 size_t buflen,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8046 PyObject *v)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8047 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8048 /* presume that the buffer is at least 2 characters long */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8049 if (PyUnicode_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8050 if (PyUnicode_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8051 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8052 buf[0] = PyUnicode_AS_UNICODE(v)[0];
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8053 }
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8054
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8055 else if (PyString_Check(v)) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8056 if (PyString_GET_SIZE(v) != 1)
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8057 goto onError;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8058 buf[0] = (Py_UNICODE)PyString_AS_STRING(v)[0];
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8059 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8060
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8061 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8062 /* Integer input truncated to a character */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8063 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8064 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8065 if (x == -1 && PyErr_Occurred())
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8066 goto onError;
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8067 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8068 if (x < 0 || x > 0x10ffff) {
28494
32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises
Walter Dörwald <walter@livinglogic.de>
parents: 28242
diff changeset
8069 PyErr_SetString(PyExc_OverflowError,
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8070 "%c arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8071 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8072 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8073 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8074 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8075 if (x < 0 || x > 0xffff) {
28494
32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises
Walter Dörwald <walter@livinglogic.de>
parents: 28242
diff changeset
8076 PyErr_SetString(PyExc_OverflowError,
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8077 "%c arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8078 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8079 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8080 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8081 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
8082 buf[0] = (Py_UNICODE) x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8083 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8084 buf[1] = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8085 return 1;
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8086
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8087 onError:
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8088 PyErr_SetString(PyExc_TypeError,
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8089 "%c requires int or char");
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8090 return -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8091 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8092
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8093 /* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8094
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8095 FORMATBUFLEN is the length of the buffer in which the floats, ints, &
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8096 chars are formatted. XXX This is a magic number. Each formatting
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8097 routine does bounds checking to ensure no overflow, but a better
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8098 solution may be to malloc a buffer of appropriate size for each
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8099 format. For now, the current solution is sufficient.
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8100 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8101 #define FORMATBUFLEN (size_t)120
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8102
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8103 PyObject *PyUnicode_Format(PyObject *format,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8104 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8105 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8106 Py_UNICODE *fmt, *res;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8107 Py_ssize_t fmtcnt, rescnt, reslen, arglen, argidx;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8108 int args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8109 PyUnicodeObject *result = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8110 PyObject *dict = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8111 PyObject *uformat;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8112
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8113 if (format == NULL || args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8114 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8115 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8116 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8117 uformat = PyUnicode_FromObject(format);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8118 if (uformat == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8119 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8120 fmt = PyUnicode_AS_UNICODE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8121 fmtcnt = PyUnicode_GET_SIZE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8122
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8123 reslen = rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8124 result = _PyUnicode_New(reslen);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8125 if (result == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8126 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8127 res = PyUnicode_AS_UNICODE(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8128
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8129 if (PyTuple_Check(args)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8130 arglen = PyTuple_Size(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8131 argidx = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8132 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8133 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8134 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8135 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8136 }
26164
63a08639dae2 Fix SF # 635969, No error "not all arguments converted"
Neal Norwitz <nnorwitz@gmail.com>
parents: 25895
diff changeset
8137 if (args->ob_type->tp_as_mapping && !PyTuple_Check(args) &&
63a08639dae2 Fix SF # 635969, No error "not all arguments converted"
Neal Norwitz <nnorwitz@gmail.com>
parents: 25895
diff changeset
8138 !PyObject_TypeCheck(args, &PyBaseString_Type))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8139 dict = args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8140
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8141 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8142 if (*fmt != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8143 if (--rescnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8144 rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8145 reslen += rescnt;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8146 if (_PyUnicode_Resize(&result, reslen) < 0)
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
8147 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8148 res = PyUnicode_AS_UNICODE(result) + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8149 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8150 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8151 *res++ = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8152 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8153 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8154 /* Got a format specifier */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8155 int flags = 0;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8156 Py_ssize_t width = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8157 int prec = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8158 Py_UNICODE c = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8159 Py_UNICODE fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8160 PyObject *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8161 PyObject *temp = NULL;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8162 Py_UNICODE *pbuf;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8163 Py_UNICODE sign;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8164 Py_ssize_t len;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8165 Py_UNICODE formatbuf[FORMATBUFLEN]; /* For format{float,int,char}() */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8166
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8167 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8168 if (*fmt == '(') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8169 Py_UNICODE *keystart;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8170 Py_ssize_t keylen;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8171 PyObject *key;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8172 int pcount = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8173
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8174 if (dict == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8175 PyErr_SetString(PyExc_TypeError,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8176 "format requires a mapping");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8177 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8178 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8179 ++fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8180 --fmtcnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8181 keystart = fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8182 /* Skip over balanced parentheses */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8183 while (pcount > 0 && --fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8184 if (*fmt == ')')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8185 --pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8186 else if (*fmt == '(')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8187 ++pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8188 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8189 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8190 keylen = fmt - keystart - 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8191 if (fmtcnt < 0 || pcount > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8192 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8193 "incomplete format key");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8194 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8195 }
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8196 #if 0
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8197 /* keys are converted to strings using UTF-8 and
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8198 then looked up since Python uses strings to hold
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8199 variables names etc. in its namespaces and we
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8200 wouldn't want to break common idioms. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8201 key = PyUnicode_EncodeUTF8(keystart,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8202 keylen,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8203 NULL);
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8204 #else
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8205 key = PyUnicode_FromUnicode(keystart, keylen);
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
8206 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8207 if (key == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8208 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8209 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8210 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8211 args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8212 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8213 args = PyObject_GetItem(dict, key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8214 Py_DECREF(key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8215 if (args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8216 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8217 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8218 args_owned = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8219 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8220 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8221 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8222 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8223 switch (c = *fmt++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8224 case '-': flags |= F_LJUST; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8225 case '+': flags |= F_SIGN; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8226 case ' ': flags |= F_BLANK; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8227 case '#': flags |= F_ALT; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8228 case '0': flags |= F_ZERO; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8229 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8230 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8231 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8232 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8233 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8234 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8235 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8236 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8237 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8238 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8239 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8240 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8241 width = PyInt_AsLong(v);
40626
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8242 if (width == -1 && PyErr_Occurred())
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8243 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8244 if (width < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8245 flags |= F_LJUST;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8246 width = -width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8247 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8248 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8249 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8250 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8251 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8252 width = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8253 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8254 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8255 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8256 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8257 if ((width*10) / 10 != width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8258 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8259 "width too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8260 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8261 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8262 width = width*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8264 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8265 if (c == '.') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8266 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8267 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8268 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8269 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8270 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8271 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8272 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8273 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8274 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8275 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8276 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8277 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8278 prec = PyInt_AsLong(v);
40626
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8279 if (prec == -1 && PyErr_Occurred())
4aee3f672f77 Merged the int/long unification branch, by very crude means (sorry Thomas!).
Guido van Rossum <guido@python.org>
parents: 40450
diff changeset
8280 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8281 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8282 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8283 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8284 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8285 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8286 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8287 prec = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8288 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8289 c = Py_CHARMASK(*fmt++);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8290 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8291 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8292 if ((prec*10) / 10 != prec) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8293 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8294 "prec too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8295 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8296 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8297 prec = prec*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8298 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8299 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8300 } /* prec */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8301 if (fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8302 if (c == 'h' || c == 'l' || c == 'L') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8303 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8304 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8305 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8306 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8307 if (fmtcnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8308 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8309 "incomplete format");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8310 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8311 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8312 if (c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8313 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8314 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8315 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8316 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8317 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8318 fill = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8319 switch (c) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8320
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8321 case '%':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8322 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8323 /* presume that buffer length is at least 1 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8324 pbuf[0] = '%';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8325 len = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8326 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8327
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8328 case 's':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8329 case 'r':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8330 if (PyUnicode_Check(v) && c == 's') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8331 temp = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8332 Py_INCREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8333 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8334 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8335 PyObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8336 if (c == 's')
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8337 temp = PyObject_Unicode(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8338 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8339 temp = PyObject_Repr(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8340 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8341 goto onError;
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8342 if (PyUnicode_Check(temp))
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8343 /* nothing to do */;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8344 else if (PyString_Check(temp)) {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8345 /* convert to string to Unicode */
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8346 unicode = PyUnicode_Decode(PyString_AS_STRING(temp),
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8347 PyString_GET_SIZE(temp),
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8348 NULL,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8349 "strict");
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8350 Py_DECREF(temp);
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8351 temp = unicode;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8352 if (temp == NULL)
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8353 goto onError;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8354 }
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8355 else {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8356 Py_DECREF(temp);
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8357 PyErr_SetString(PyExc_TypeError,
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8358 "%s argument has non-string str()");
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8359 goto onError;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8360 }
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
8361 }
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8362 pbuf = PyUnicode_AS_UNICODE(temp);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8363 len = PyUnicode_GET_SIZE(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8364 if (prec >= 0 && len > prec)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8365 len = prec;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8366 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8367
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8368 case 'i':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8369 case 'd':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8370 case 'u':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8371 case 'o':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8372 case 'x':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8373 case 'X':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8374 if (c == 'i')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8375 c = 'd';
15455
d0146d128897 Fox for SF bug #123859: %[duxXo] long formats inconsistent.
Tim Peters <tim.peters@gmail.com>
parents: 15001
diff changeset
8376 if (PyLong_Check(v)) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8377 temp = formatlong(v, flags, prec, c);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8378 if (!temp)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8379 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8380 pbuf = PyUnicode_AS_UNICODE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8381 len = PyUnicode_GET_SIZE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8382 sign = 1;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8383 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8384 else {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8385 pbuf = formatbuf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8386 len = formatint(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8387 flags, prec, c, v);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8388 if (len < 0)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8389 goto onError;
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
8390 sign = 1;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8391 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8392 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8393 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8394 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8395
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8396 case 'e':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8397 case 'E':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8398 case 'f':
30216
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
8399 case 'F':
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8400 case 'g':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8401 case 'G':
30216
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
8402 if (c == 'F')
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
8403 c = 'f';
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8404 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8405 len = formatfloat(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8406 flags, prec, c, v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8407 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8408 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8409 sign = 1;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8410 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8411 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8412 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8413
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8414 case 'c':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8415 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8416 len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8417 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8418 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8419 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8420
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8421 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8422 PyErr_Format(PyExc_ValueError,
15579
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the
Andrew M. Kuchling <amk@amk.ca>
parents: 15455
diff changeset
8423 "unsupported format character '%c' (0x%x) "
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
8424 "at index %zd",
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
8425 (31<=c && c<=126) ? (char)c : '?',
25375
4df0e691f4da Add cast to avoid compiler warning.
Marc-André Lemburg <mal@egenix.com>
parents: 25248
diff changeset
8426 (int)c,
40450
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
8427 (Py_ssize_t)(fmt - 1 -
cc992d75d5b3 Four months of trunk changes (including a few releases...)
Thomas Wouters <thomas@python.org>
parents: 39610
diff changeset
8428 PyUnicode_AS_UNICODE(uformat)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8429 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8430 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8431 if (sign) {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8432 if (*pbuf == '-' || *pbuf == '+') {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
8433 sign = *pbuf++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8434 len--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8435 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8436 else if (flags & F_SIGN)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8437 sign = '+';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8438 else if (flags & F_BLANK)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8439 sign = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8440 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8441 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8442 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8443 if (width < len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8444 width = len;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
8445 if (rescnt - (sign != 0) < width) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8446 reslen -= rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8447 rescnt = width + fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8448 reslen += rescnt;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
8449 if (reslen < 0) {
36877
e9a7a2784e17 SF #1444030: Fix several potential defects found by Coverity.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36675
diff changeset
8450 Py_XDECREF(temp);
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8451 PyErr_NoMemory();
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8452 goto onError;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
8453 }
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8454 if (_PyUnicode_Resize(&result, reslen) < 0) {
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8455 Py_XDECREF(temp);
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8456 goto onError;
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8457 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8458 res = PyUnicode_AS_UNICODE(result)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8459 + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8460 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8461 if (sign) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8462 if (fill != ' ')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8463 *res++ = sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8464 rescnt--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8465 if (width > len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8466 width--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8467 }
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8468 if ((flags & F_ALT) && (c == 'x' || c == 'X' || c == 'o')) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8469 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8470 assert(pbuf[1] == c);
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8471 if (fill != ' ') {
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8472 *res++ = *pbuf++;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8473 *res++ = *pbuf++;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8474 }
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8475 rescnt -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8476 width -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8477 if (width < 0)
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8478 width = 0;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8479 len -= 2;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8480 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8481 if (width > len && !(flags & F_LJUST)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8482 do {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8483 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8484 *res++ = fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8485 } while (--width > len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8486 }
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8487 if (fill == ' ') {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8488 if (sign)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8489 *res++ = sign;
42084
229c28cb9afb Merged revisions 55817-55961 via svnmerge from
Guido van Rossum <guido@python.org>
parents: 42073
diff changeset
8490 if ((flags & F_ALT) && (c == 'x' || c == 'X' || c == 'o')) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8491 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
8492 assert(pbuf[1] == c);
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8493 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8494 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8495 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
8496 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8497 Py_UNICODE_COPY(res, pbuf, len);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8498 res += len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8499 rescnt -= len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8500 while (--width >= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8501 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8502 *res++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8503 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8504 if (dict && (argidx < arglen) && c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8505 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
8506 "not all arguments converted during string formatting");
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8507 Py_XDECREF(temp);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8508 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8509 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8510 Py_XDECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8511 } /* '%' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8512 } /* until end */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8513 if (argidx < arglen && !dict) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8514 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
8515 "not all arguments converted during string formatting");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8516 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8517 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8518
36945
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8519 if (_PyUnicode_Resize(&result, reslen - rescnt) < 0)
0f19a6fc9756 - Reindent a confusingly indented piece of code (no intended code changes
Thomas Wouters <thomas@python.org>
parents: 36924
diff changeset
8520 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8521 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8522 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8523 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8524 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8525 return (PyObject *)result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8526
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8527 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8528 Py_XDECREF(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8529 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8530 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8531 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8532 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8533 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8534 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8535
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8536 static PyBufferProcs unicode_as_buffer = {
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8537 (readbufferproc) unicode_buffer_getreadbuf,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8538 (writebufferproc) unicode_buffer_getwritebuf,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8539 (segcountproc) unicode_buffer_getsegcount,
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8540 (charbufferproc) unicode_buffer_getcharbuf,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8541 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8542
24280
1598a1e473e2 staticforward bites the dust.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24071
diff changeset
8543 static PyObject *
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8544 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8545
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8546 static PyObject *
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8547 unicode_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8548 {
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8549 PyObject *x = NULL;
42184
bf5d04e0da4c Make test_descr.py pass. Had to disable a few tests, remove references
Guido van Rossum <guido@python.org>
parents: 42123
diff changeset
8550 static char *kwlist[] = {"object", "encoding", "errors", 0};
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8551 char *encoding = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8552 char *errors = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8553
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8554 if (type != &PyUnicode_Type)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8555 return unicode_subtype_new(type, args, kwds);
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8556 if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:unicode",
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8557 kwlist, &x, &encoding, &errors))
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8558 return NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8559 if (x == NULL)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8560 return (PyObject *)_PyUnicode_New(0);
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
8561 if (encoding == NULL && errors == NULL)
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
8562 return PyObject_Unicode(x);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
8563 else
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8564 return PyUnicode_FromEncodedObject(x, encoding, errors);
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8565 }
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8566
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8567 static PyObject *
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8568 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8569 {
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8570 PyUnicodeObject *tmp, *pnew;
36501
7fdc639bc5b4 Merge ssize_t branch.
Martin v. Löwis <martin@v.loewis.de>
parents: 36261
diff changeset
8571 Py_ssize_t n;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8572
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8573 assert(PyType_IsSubtype(type, &PyUnicode_Type));
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8574 tmp = (PyUnicodeObject *)unicode_new(&PyUnicode_Type, args, kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8575 if (tmp == NULL)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8576 return NULL;
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8577 assert(PyUnicode_Check(tmp));
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8578 pnew = (PyUnicodeObject *) type->tp_alloc(type, n = tmp->length);
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8579 if (pnew == NULL) {
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8580 Py_DECREF(tmp);
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8581 return NULL;
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8582 }
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8583 pnew->str = PyMem_NEW(Py_UNICODE, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8584 if (pnew->str == NULL) {
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8585 _Py_ForgetReference((PyObject *)pnew);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
8586 PyObject_Del(pnew);
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
8587 Py_DECREF(tmp);
27826
7dffcc498c45 Add more missing PyErr_NoMemory() after failled memory allocs
Neal Norwitz <nnorwitz@gmail.com>
parents: 27774
diff changeset
8588 return PyErr_NoMemory();
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8589 }
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8590 Py_UNICODE_COPY(pnew->str, tmp->str, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8591 pnew->length = n;
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8592 pnew->hash = tmp->hash;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8593 Py_DECREF(tmp);
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
8594 return (PyObject *)pnew;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8595 }
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
8596
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
8597 PyDoc_STRVAR(unicode_doc,
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8598 "unicode(string [, encoding[, errors]]) -> object\n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8599 \n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8600 Create a new Unicode object from the given encoded string.\n\
24392
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
8601 encoding defaults to the current default string encoding.\n\
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
8602 errors can be 'strict', 'replace' or 'ignore' and defaults to 'strict'.");
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8603
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8604 static PyObject *unicode_iter(PyObject *seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8605
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8606 PyTypeObject PyUnicode_Type = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8607 PyObject_HEAD_INIT(&PyType_Type)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8608 0, /* ob_size */
41585
825d966338ea Rename 'unicode' to 'str' in its tp_name field. Rename 'str' to 'str8'.
Guido van Rossum <guido@python.org>
parents: 41558
diff changeset
8609 "str", /* tp_name */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8610 sizeof(PyUnicodeObject), /* tp_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8611 0, /* tp_itemsize */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8612 /* Slots */
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
8613 (destructor)unicode_dealloc, /* tp_dealloc */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8614 0, /* tp_print */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8615 0, /* tp_getattr */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8616 0, /* tp_setattr */
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
8617 0, /* tp_compare */
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
8618 unicode_repr, /* tp_repr */
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
8619 &unicode_as_number, /* tp_as_number */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8620 &unicode_as_sequence, /* tp_as_sequence */
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
8621 &unicode_as_mapping, /* tp_as_mapping */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8622 (hashfunc) unicode_hash, /* tp_hash*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8623 0, /* tp_call*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8624 (reprfunc) unicode_str, /* tp_str */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8625 PyObject_GenericGetAttr, /* tp_getattro */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8626 0, /* tp_setattro */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8627 &unicode_as_buffer, /* tp_as_buffer */
40909
f2f0b41f270d Merged revisions 53875-53911 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 40626
diff changeset
8628 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE |
f2f0b41f270d Merged revisions 53875-53911 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 40626
diff changeset
8629 Py_TPFLAGS_UNICODE_SUBCLASS, /* tp_flags */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8630 unicode_doc, /* tp_doc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8631 0, /* tp_traverse */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8632 0, /* tp_clear */
39610
3674f20ac781 Merge current trunk into p3yk. This includes the PyNumber_Index API change,
Thomas Wouters <thomas@python.org>
parents: 39544
diff changeset
8633 PyUnicode_RichCompare, /* tp_richcompare */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8634 0, /* tp_weaklistoffset */
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8635 unicode_iter, /* tp_iter */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8636 0, /* tp_iternext */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8637 unicode_methods, /* tp_methods */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8638 0, /* tp_members */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8639 0, /* tp_getset */
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
8640 &PyBaseString_Type, /* tp_base */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8641 0, /* tp_dict */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8642 0, /* tp_descr_get */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8643 0, /* tp_descr_set */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8644 0, /* tp_dictoffset */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8645 0, /* tp_init */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8646 0, /* tp_alloc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
8647 unicode_new, /* tp_new */
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
8648 PyObject_Del, /* tp_free */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8649 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8650
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8651 /* Initialize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8652
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
8653 void _PyUnicode_Init(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8654 {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8655 int i;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8656
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8657 /* XXX - move this array to unicodectype.c ? */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8658 Py_UNICODE linebreak[] = {
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8659 0x000A, /* LINE FEED */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8660 0x000D, /* CARRIAGE RETURN */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8661 0x001C, /* FILE SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8662 0x001D, /* GROUP SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8663 0x001E, /* RECORD SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8664 0x0085, /* NEXT LINE */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8665 0x2028, /* LINE SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8666 0x2029, /* PARAGRAPH SEPARATOR */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8667 };
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8668
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
8669 /* Init the implementation */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8670 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8671 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8672 unicode_empty = _PyUnicode_New(0);
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8673 if (!unicode_empty)
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8674 return;
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8675
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8676 for (i = 0; i < 256; i++)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8677 unicode_latin1[i] = NULL;
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
8678 if (PyType_Ready(&PyUnicode_Type) < 0)
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
8679 Py_FatalError("Can't initialize 'unicode'");
38453
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8680
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8681 /* initialize the linebreak bloom filter */
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8682 bloom_linebreak = make_bloom_mask(
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8683 linebreak, sizeof(linebreak) / sizeof(linebreak[0])
29ae5e0fc348 Much-needed merge (using svnmerge.py this time) of trunk changes into p3yk.
Thomas Wouters <thomas@python.org>
parents: 37879
diff changeset
8684 );
39442
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8685
654c380cf8b9 Merged revisions 46753-51188 via svnmerge from
Thomas Wouters <thomas@python.org>
parents: 39222
diff changeset
8686 PyType_Ready(&EncodingMapType);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8687 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8688
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8689 /* Finalize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8690
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8691 void
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
8692 _PyUnicode_Fini(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8693 {
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
8694 PyUnicodeObject *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8695 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8696
14998
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
8697 Py_XDECREF(unicode_empty);
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
8698 unicode_empty = NULL;
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
8699
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8700 for (i = 0; i < 256; i++) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8701 if (unicode_latin1[i]) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8702 Py_DECREF(unicode_latin1[i]);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8703 unicode_latin1[i] = NULL;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8704 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8705 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
8706
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
8707 for (u = unicode_freelist; u != NULL;) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8708 PyUnicodeObject *v = u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8709 u = *(PyUnicodeObject **)u;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
8710 if (v->str)
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
8711 PyMem_DEL(v->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
8712 Py_XDECREF(v->defenc);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
8713 PyObject_Del(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8714 }
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8715 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
8716 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8717 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
8718
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8719 void
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8720 PyUnicode_InternInPlace(PyObject **p)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8721 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8722 register PyUnicodeObject *s = (PyUnicodeObject *)(*p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8723 PyObject *t;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8724 if (s == NULL || !PyUnicode_Check(s))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8725 Py_FatalError(
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8726 "PyUnicode_InternInPlace: unicode strings only please!");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8727 /* If it's a subclass, we don't really know what putting
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8728 it in the interned dict might do. */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8729 if (!PyUnicode_CheckExact(s))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8730 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8731 if (PyUnicode_CHECK_INTERNED(s))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8732 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8733 if (interned == NULL) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8734 interned = PyDict_New();
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8735 if (interned == NULL) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8736 PyErr_Clear(); /* Don't leave an exception */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8737 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8738 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8739 }
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8740 /* It might be that the GetItem call fails even
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8741 though the key is present in the dictionary,
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8742 namely when this happens during a stack overflow. */
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8743 Py_ALLOW_RECURSION
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8744 t = PyDict_GetItem(interned, (PyObject *)s);
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8745 Py_END_ALLOW_RECURSION
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8746
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8747 if (t) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8748 Py_INCREF(t);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8749 Py_DECREF(*p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8750 *p = t;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8751 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8752 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8753
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8754 PyThreadState_GET()->recursion_critical = 1;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8755 if (PyDict_SetItem(interned, (PyObject *)s, (PyObject *)s) < 0) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8756 PyErr_Clear();
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8757 PyThreadState_GET()->recursion_critical = 0;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8758 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8759 }
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8760 PyThreadState_GET()->recursion_critical = 0;
41876
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8761 /* The two references in interned are not counted by refcnt.
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8762 The deallocator will take care of this */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8763 s->ob_refcnt -= 2;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8764 PyUnicode_CHECK_INTERNED(s) = SSTATE_INTERNED_MORTAL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8765 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8766
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8767 void
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8768 PyUnicode_InternImmortal(PyObject **p)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8769 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8770 PyUnicode_InternInPlace(p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8771 if (PyUnicode_CHECK_INTERNED(*p) != SSTATE_INTERNED_IMMORTAL) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8772 PyUnicode_CHECK_INTERNED(*p) = SSTATE_INTERNED_IMMORTAL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8773 Py_INCREF(*p);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8774 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8775 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8776
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8777 PyObject *
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8778 PyUnicode_InternFromString(const char *cp)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8779 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8780 PyObject *s = PyUnicode_FromString(cp);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8781 if (s == NULL)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8782 return NULL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8783 PyUnicode_InternInPlace(&s);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8784 return s;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8785 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8786
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8787 void _Py_ReleaseInternedUnicodeStrings(void)
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8788 {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8789 PyObject *keys;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8790 PyUnicodeObject *s;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8791 Py_ssize_t i, n;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8792 Py_ssize_t immortal_size = 0, mortal_size = 0;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8793
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8794 if (interned == NULL || !PyDict_Check(interned))
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8795 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8796 keys = PyDict_Keys(interned);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8797 if (keys == NULL || !PyList_Check(keys)) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8798 PyErr_Clear();
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8799 return;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8800 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8801
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8802 /* Since _Py_ReleaseInternedUnicodeStrings() is intended to help a leak
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8803 detector, interned unicode strings are not forcibly deallocated;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8804 rather, we give them their stolen references back, and then clear
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8805 and DECREF the interned dict. */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8806
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8807 n = PyList_GET_SIZE(keys);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8808 fprintf(stderr, "releasing %" PY_FORMAT_SIZE_T "d interned strings\n",
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8809 n);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8810 for (i = 0; i < n; i++) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8811 s = (PyUnicodeObject *) PyList_GET_ITEM(keys, i);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8812 switch (s->state) {
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8813 case SSTATE_NOT_INTERNED:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8814 /* XXX Shouldn't happen */
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8815 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8816 case SSTATE_INTERNED_IMMORTAL:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8817 s->ob_refcnt += 1;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8818 immortal_size += s->length;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8819 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8820 case SSTATE_INTERNED_MORTAL:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8821 s->ob_refcnt += 2;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8822 mortal_size += s->length;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8823 break;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8824 default:
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8825 Py_FatalError("Inconsistent interned string state.");
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8826 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8827 s->state = SSTATE_NOT_INTERNED;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8828 }
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8829 fprintf(stderr, "total size of all interned strings: "
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8830 "%" PY_FORMAT_SIZE_T "d/%" PY_FORMAT_SIZE_T "d "
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8831 "mortal/immortal\n", mortal_size, immortal_size);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8832 Py_DECREF(keys);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8833 PyDict_Clear(interned);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8834 Py_DECREF(interned);
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8835 interned = NULL;
dcb43f40390b Add interning of unicode strings by copying the functionality from
Walter Dörwald <walter@livinglogic.de>
parents: 41861
diff changeset
8836 }
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8837
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8838
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8839 /********************* Unicode Iterator **************************/
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8840
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8841 typedef struct {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8842 PyObject_HEAD
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8843 Py_ssize_t it_index;
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8844 PyUnicodeObject *it_seq; /* Set to NULL when iterator is exhausted */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8845 } unicodeiterobject;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8846
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8847 static void
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8848 unicodeiter_dealloc(unicodeiterobject *it)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8849 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8850 _PyObject_GC_UNTRACK(it);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8851 Py_XDECREF(it->it_seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8852 PyObject_GC_Del(it);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8853 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8854
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8855 static int
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8856 unicodeiter_traverse(unicodeiterobject *it, visitproc visit, void *arg)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8857 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8858 Py_VISIT(it->it_seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8859 return 0;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8860 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8861
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8862 static PyObject *
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8863 unicodeiter_next(unicodeiterobject *it)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8864 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8865 PyUnicodeObject *seq;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8866 PyObject *item;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8867
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8868 assert(it != NULL);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8869 seq = it->it_seq;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8870 if (seq == NULL)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8871 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8872 assert(PyUnicode_Check(seq));
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8873
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8874 if (it->it_index < PyUnicode_GET_SIZE(seq)) {
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8875 item = PyUnicode_FromUnicode(
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8876 PyUnicode_AS_UNICODE(seq)+it->it_index, 1);
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8877 if (item != NULL)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8878 ++it->it_index;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8879 return item;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8880 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8881
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8882 Py_DECREF(seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8883 it->it_seq = NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8884 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8885 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8886
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8887 static PyObject *
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8888 unicodeiter_len(unicodeiterobject *it)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8889 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8890 Py_ssize_t len = 0;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8891 if (it->it_seq)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8892 len = PyUnicode_GET_SIZE(it->it_seq) - it->it_index;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8893 return PyInt_FromSsize_t(len);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8894 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8895
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8896 PyDoc_STRVAR(length_hint_doc, "Private method returning an estimate of len(list(it)).");
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8897
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8898 static PyMethodDef unicodeiter_methods[] = {
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8899 {"__length_hint__", (PyCFunction)unicodeiter_len, METH_NOARGS,
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8900 length_hint_doc},
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8901 {NULL, NULL} /* sentinel */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8902 };
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8903
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8904 PyTypeObject PyUnicodeIter_Type = {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8905 PyObject_HEAD_INIT(&PyType_Type)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8906 0, /* ob_size */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8907 "unicodeiterator", /* tp_name */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8908 sizeof(unicodeiterobject), /* tp_basicsize */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8909 0, /* tp_itemsize */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8910 /* methods */
39544
01034018e8b2 Make the it_index field in the str/unicode iterators Py_ssize_t's.
Guido van Rossum <guido@python.org>
parents: 39536
diff changeset
8911 (destructor)unicodeiter_dealloc, /* tp_dealloc */
39536
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8912 0, /* tp_print */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8913 0, /* tp_getattr */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8914 0, /* tp_setattr */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8915 0, /* tp_compare */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8916 0, /* tp_repr */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8917 0, /* tp_as_number */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8918 0, /* tp_as_sequence */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8919 0, /* tp_as_mapping */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8920 0, /* tp_hash */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8921 0, /* tp_call */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8922 0, /* tp_str */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8923 PyObject_GenericGetAttr, /* tp_getattro */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8924 0, /* tp_setattro */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8925 0, /* tp_as_buffer */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8926 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_HAVE_GC,/* tp_flags */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8927 0, /* tp_doc */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8928 (traverseproc)unicodeiter_traverse, /* tp_traverse */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8929 0, /* tp_clear */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8930 0, /* tp_richcompare */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8931 0, /* tp_weaklistoffset */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8932 PyObject_SelfIter, /* tp_iter */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8933 (iternextfunc)unicodeiter_next, /* tp_iternext */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8934 unicodeiter_methods, /* tp_methods */
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8935 0,
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8936 };
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8937
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8938 static PyObject *
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8939 unicode_iter(PyObject *seq)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8940 {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8941 unicodeiterobject *it;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8942
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8943 if (!PyUnicode_Check(seq)) {
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8944 PyErr_BadInternalCall();
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8945 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8946 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8947 it = PyObject_GC_New(unicodeiterobject, &PyUnicodeIter_Type);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8948 if (it == NULL)
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8949 return NULL;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8950 it->it_index = 0;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8951 Py_INCREF(seq);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8952 it->it_seq = (PyUnicodeObject *)seq;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8953 _PyObject_GC_TRACK(it);
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8954 return (PyObject *)it;
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8955 }
8f5b9982beba Completely get rid of PyClass and PyInstance.
Guido van Rossum <guido@python.org>
parents: 39442
diff changeset
8956
42013
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8957 size_t
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8958 Py_UNICODE_strlen(const Py_UNICODE *u)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8959 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8960 int res = 0;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8961 while(*u++)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8962 res++;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8963 return res;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8964 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8965
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8966 Py_UNICODE*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8967 Py_UNICODE_strcpy(Py_UNICODE *s1, const Py_UNICODE *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8968 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8969 Py_UNICODE *u = s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8970 while ((*u++ = *s2++));
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8971 return s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8972 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8973
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8974 Py_UNICODE*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8975 Py_UNICODE_strncpy(Py_UNICODE *s1, const Py_UNICODE *s2, size_t n)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8976 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8977 Py_UNICODE *u = s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8978 while ((*u++ = *s2++))
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8979 if (n-- == 0)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8980 break;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8981 return s1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8982 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8983
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8984 int
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8985 Py_UNICODE_strcmp(const Py_UNICODE *s1, const Py_UNICODE *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8986 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8987 while (*s1 && *s2 && *s1 == *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8988 s1++, s2++;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8989 if (*s1 && *s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8990 return (*s1 < *s2) ? -1 : +1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8991 if (*s1)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8992 return 1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8993 if (*s2)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8994 return -1;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8995 return 0;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8996 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8997
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8998 Py_UNICODE*
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
8999 Py_UNICODE_strchr(const Py_UNICODE *s, Py_UNICODE c)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9000 {
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9001 const Py_UNICODE *p;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9002 for (p = s; *p; p++)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9003 if (*p == c)
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9004 return (Py_UNICODE*)p;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9005 return NULL;
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9006 }
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9007
cd125fe83051 Make identifiers str (not str8) objects throughout.
Martin v. Löwis <martin@v.loewis.de>
parents: 41954
diff changeset
9008
37879
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9009 #ifdef __cplusplus
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9010 }
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9011 #endif
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9012
f731859e9b3b Merge p3yk branch with the trunk up to revision 45595. This breaks a fair
Thomas Wouters <thomas@python.org>
parents: 37191
diff changeset
9013
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9014 /*
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9015 Local variables:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9016 c-basic-offset: 4
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9017 indent-tabs-mode: nil
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9018 End:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
9019 */