Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodeobject.c @ 39953:84afa366eee7 2.4

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
[Backport rev. 47171 by neal.norwitz] Another problem reported by Coverity. Backport candidate.
author Andrew M. Kuchling <amk@amk.ca>
date Tue, 03 Oct 2006 12:58:52 +0000
parents 2632c9af64ab
children a9b1f404e88b
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1 /*
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 Unicode implementation based on original code by Fredrik Lundh,
12646
17d1a3f1f085 Minimal change so I can add the rest of MAL's checkin message:
Fred Drake <fdrake@acm.org>
parents: 12645
diff changeset
4 modified by Marc-Andre Lemburg <mal@lemburg.com> according to the
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 Unicode Integration Proposal (see file Misc/unicode.txt).
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13761
diff changeset
7 Copyright (c) Corporation for National Research Initiatives.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
16029
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
9 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
10 The original string type implementation is:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
11
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
12 Copyright (c) 1999 by Secret Labs AB
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
13 Copyright (c) 1999 by Fredrik Lundh
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
14
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
15 By obtaining, using, and/or copying this software and/or its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
16 associated documentation, you agree that you have read, understood,
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
17 and will comply with the following terms and conditions:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
18
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
19 Permission to use, copy, modify, and distribute this software and its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
20 associated documentation for any purpose and without fee is hereby
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
21 granted, provided that the above copyright notice appears in all
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
22 copies, and that both that copyright notice and this permission notice
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
23 appear in supporting documentation, and that the name of Secret Labs
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
24 AB or the author not be used in advertising or publicity pertaining to
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
25 distribution of the software without specific, written prior
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
26 permission.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
27
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
28 SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
29 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
30 FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
31 ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
32 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
33 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
34 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
35 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
36
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
37 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
38
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
39 #include "Python.h"
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
40
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 #include "unicodeobject.h"
13209
b11e0a898de0 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13196
diff changeset
42 #include "ucnhash.h"
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
43
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
44 #ifdef MS_WINDOWS
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
45 #include <windows.h>
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
46 #endif
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
47
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
48 /* Limit for the Unicode object free list */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
49
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
50 #define MAX_UNICODE_FREELIST_SIZE 1024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
51
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 /* Limit for the Unicode object free list stay alive optimization.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
53
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 The implementation will keep allocated Unicode memory intact for
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 all objects on the free list having a size less than this
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
56 limit. This reduces malloc() overhead for small Unicode objects.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
57
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
58 At worst this will result in MAX_UNICODE_FREELIST_SIZE *
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
59 (sizeof(PyUnicodeObject) + KEEPALIVE_SIZE_LIMIT +
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
60 malloc()-overhead) bytes of unused garbage.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
61
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 Setting the limit to 0 effectively turns the feature off.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
63
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
64 Note: This is an experimental feature ! If you get core dumps when
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
65 using Unicode objects, turn this feature off.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
66
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
67 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
68
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
69 #define KEEPALIVE_SIZE_LIMIT 9
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
70
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 /* Endianness switches; defaults to little endian */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
72
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
73 #ifdef WORDS_BIGENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
74 # define BYTEORDER_IS_BIG_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 # define BYTEORDER_IS_LITTLE_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
78
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
79 /* --- Globals ------------------------------------------------------------
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
80
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
81 The globals are initialized by the _PyUnicode_Init() API and should
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
82 not be used before calling that API.
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
83
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
84 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
85
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
86 /* Free list for Unicode objects */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
87 static PyUnicodeObject *unicode_freelist;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
88 static int unicode_freelist_size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
89
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
90 /* The empty Unicode object is shared to improve performance. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
91 static PyUnicodeObject *unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
92
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
93 /* Single character Unicode strings in the Latin-1 range are being
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
94 shared as well. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
95 static PyUnicodeObject *unicode_latin1[256];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
96
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
97 /* Default encoding to use and assume when NULL is passed as encoding
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
98 parameter; it is initialized by _PyUnicode_Init().
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
99
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
100 Always use the PyUnicode_SetDefaultEncoding() and
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
101 PyUnicode_GetDefaultEncoding() APIs to access this global.
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
102
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
103 */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
104 static char unicode_default_encoding[100];
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
105
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
106 Py_UNICODE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
107 PyUnicode_GetMax(void)
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
108 {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
109 #ifdef Py_UNICODE_WIDE
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
110 return 0x10FFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
111 #else
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
112 /* This is actually an illegal character, so it should
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
113 not be passed to unichr. */
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
114 return 0xFFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
115 #endif
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
116 }
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
117
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
118 /* --- Unicode Object ----------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
119
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
120 static
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
121 int unicode_resize(register PyUnicodeObject *unicode,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
122 int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
124 void *oldstr;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
125
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
126 /* Shortcut if there's nothing much to do. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 if (unicode->length == length)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
128 goto reset;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
129
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
130 /* Resizing shared object (unicode_empty or single character
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
131 objects) in-place is not allowed. Use PyUnicode_Resize()
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
132 instead ! */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
133 if (unicode == unicode_empty ||
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
134 (unicode->length == 1 &&
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
135 /* MvL said unicode->str[] may be signed. Python generally assumes
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
136 * an int contains at least 32 bits, and we don't use more than
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
137 * 32 bits even in a UCS4 build, so casting to unsigned int should
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
138 * be correct.
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
139 */
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
140 (unsigned int)unicode->str[0] < 256U &&
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
141 unicode_latin1[unicode->str[0]] == unicode)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
142 PyErr_SetString(PyExc_SystemError,
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
143 "can't resize shared unicode objects");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
144 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
145 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
146
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
147 /* We allocate one more byte to make sure the string is
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
148 Ux0000 terminated -- XXX is this needed ? */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
149 oldstr = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
150 PyMem_RESIZE(unicode->str, Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
151 if (!unicode->str) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
152 unicode->str = oldstr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
153 PyErr_NoMemory();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
154 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
155 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
156 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
157 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
158
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
159 reset:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
160 /* Reset the object caches */
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
161 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
162 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
163 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
164 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
165 unicode->hash = -1;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
166
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
167 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
168 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
169
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
170 /* We allocate one more byte to make sure the string is
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
171 Ux0000 terminated -- XXX is this needed ?
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
172
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
173 XXX This allocator could further be enhanced by assuring that the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
174 free list never reduces its size below 1.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
175
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
176 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
177
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
178 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
179 PyUnicodeObject *_PyUnicode_New(int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
180 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
181 register PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
182
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
183 /* Optimization fo empty strings */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
184 if (length == 0 && unicode_empty != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
185 Py_INCREF(unicode_empty);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
186 return unicode_empty;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
187 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
188
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
189 /* Unicode freelist & memory allocation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
190 if (unicode_freelist) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
191 unicode = unicode_freelist;
12939
88004f25dc25 Vladimir MARANGOZOV <Vladimir.Marangozov@inrialpes.fr>:
Marc-André Lemburg <mal@egenix.com>
parents: 12924
diff changeset
192 unicode_freelist = *(PyUnicodeObject **)unicode;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
193 unicode_freelist_size--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
194 if (unicode->str) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
195 /* Keep-Alive optimization: we only upsize the buffer,
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
196 never downsize it. */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
197 if ((unicode->length < length) &&
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
198 unicode_resize(unicode, length) < 0) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
199 PyMem_DEL(unicode->str);
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
200 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
202 }
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
203 else {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
204 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
205 }
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
206 PyObject_INIT(unicode, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
207 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
208 else {
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
209 unicode = PyObject_New(PyUnicodeObject, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
210 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
211 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
212 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
213 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
214
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
215 if (!unicode->str) {
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
216 PyErr_NoMemory();
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
217 goto onError;
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
218 }
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
219 /* Initialize the first element to guard against cases where
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
220 * the caller fails before initializing str -- unicode_resize()
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
221 * reads str[0], and the Keep-Alive optimization can keep memory
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
222 * allocated for str alive across a call to unicode_dealloc(unicode).
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
223 * We don't want unicode_resize to read uninitialized memory in
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
224 * that case.
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
225 */
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
226 unicode->str[0] = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
227 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
228 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
229 unicode->hash = -1;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
230 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
231 return unicode;
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
232
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
233 onError:
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
234 _Py_ForgetReference((PyObject *)unicode);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
235 PyObject_Del(unicode);
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
236 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
237 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
239 static
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
240 void unicode_dealloc(register PyUnicodeObject *unicode)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
241 {
21262
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
242 if (PyUnicode_CheckExact(unicode) &&
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
243 unicode_freelist_size < MAX_UNICODE_FREELIST_SIZE) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
244 /* Keep-Alive optimization */
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
245 if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
246 PyMem_DEL(unicode->str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
247 unicode->str = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
248 unicode->length = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
249 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
250 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
251 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
252 unicode->defenc = NULL;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
253 }
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
254 /* Add to free list */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
255 *(PyUnicodeObject **)unicode = unicode_freelist;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
256 unicode_freelist = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
257 unicode_freelist_size++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
258 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
259 else {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
260 PyMem_DEL(unicode->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
261 Py_XDECREF(unicode->defenc);
21262
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
262 unicode->ob_type->tp_free((PyObject *)unicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
264 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
265
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
266 int PyUnicode_Resize(PyObject **unicode, int length)
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
267 {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
268 register PyUnicodeObject *v;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
269
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
270 /* Argument checks */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
271 if (unicode == NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
272 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
273 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
274 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
275 v = (PyUnicodeObject *)*unicode;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
276 if (v == NULL || !PyUnicode_Check(v) || v->ob_refcnt != 1 || length < 0) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
277 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
278 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
279 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
280
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
281 /* Resizing unicode_empty and single character objects is not
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
282 possible since these are being shared. We simply return a fresh
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
283 copy with the same Unicode content. */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
284 if (v->length != length &&
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
285 (v == unicode_empty || v->length == 1)) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
286 PyUnicodeObject *w = _PyUnicode_New(length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
287 if (w == NULL)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
288 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
289 Py_UNICODE_COPY(w->str, v->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
290 length < v->length ? length : v->length);
28242
fe1d571c3f80 Sf patch #700047: unicode object leaks refcount on resizing
Raymond Hettinger <python@rcn.com>
parents: 27826
diff changeset
291 Py_DECREF(*unicode);
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
292 *unicode = (PyObject *)w;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
293 return 0;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
294 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
295
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
296 /* Note that we don't have to modify *unicode for unshared Unicode
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
297 objects, since we can modify them in-place. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
298 return unicode_resize(v, length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
299 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
300
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
301 /* Internal API for use in unicodeobject.c only ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
302 #define _PyUnicode_Resize(unicodevar, length) \
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
303 PyUnicode_Resize(((PyObject **)(unicodevar)), length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
304
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
305 PyObject *PyUnicode_FromUnicode(const Py_UNICODE *u,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
306 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
307 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
308 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
309
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
310 /* If the Unicode data is known at construction time, we can apply
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
311 some optimizations which share commonly used objects. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
312 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
313
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
314 /* Optimization for empty strings */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
315 if (size == 0 && unicode_empty != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
316 Py_INCREF(unicode_empty);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
317 return (PyObject *)unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
318 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
319
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
320 /* Single character Unicode objects in the Latin-1 range are
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
321 shared when using this constructor */
38606
30605bf95ef8 Don't crash on Py_UNICODE values < 0. Fixes #1454485.
Martin v. Löwis <martin@v.loewis.de>
parents: 38183
diff changeset
322 /* XXX In Python 2.4, Py_UNICODE can, unfortunately, be a signed
30605bf95ef8 Don't crash on Py_UNICODE values < 0. Fixes #1454485.
Martin v. Löwis <martin@v.loewis.de>
parents: 38183
diff changeset
323 type. */
30605bf95ef8 Don't crash on Py_UNICODE values < 0. Fixes #1454485.
Martin v. Löwis <martin@v.loewis.de>
parents: 38183
diff changeset
324 if (size == 1 && *u >= 0 && *u < 256) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
325 unicode = unicode_latin1[*u];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
326 if (!unicode) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
327 unicode = _PyUnicode_New(1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
328 if (!unicode)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
329 return NULL;
18051
e897550b4748 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode.
Marc-André Lemburg <mal@egenix.com>
parents: 17995
diff changeset
330 unicode->str[0] = *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
331 unicode_latin1[*u] = unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
332 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
333 Py_INCREF(unicode);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
334 return (PyObject *)unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
335 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
336 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
337
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
338 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
339 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
340 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
341
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
342 /* Copy the Unicode data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
343 if (u != NULL)
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
344 Py_UNICODE_COPY(unicode->str, u, size);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
345
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
346 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
347 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
348
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
349 #ifdef HAVE_WCHAR_H
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
350
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
351 PyObject *PyUnicode_FromWideChar(register const wchar_t *w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
352 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
353 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
354 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
355
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
356 if (w == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
357 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
358 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
359 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
360
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
361 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
362 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
363 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
364
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
365 /* Copy the wchar_t data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
366 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
367 memcpy(unicode->str, w, size * sizeof(wchar_t));
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
368 #else
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
369 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
370 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
371 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
372 u = PyUnicode_AS_UNICODE(unicode);
33766
1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
Marc-André Lemburg <mal@egenix.com>
parents: 33499
diff changeset
373 for (i = size; i > 0; i--)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
374 *u++ = *w++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
375 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
376 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
377
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
378 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
379 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
380
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
381 int PyUnicode_AsWideChar(PyUnicodeObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
382 register wchar_t *w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
383 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
384 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
385 if (unicode == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
386 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
387 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
388 }
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
389
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
390 /* If possible, try to copy the 0-termination as well */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
391 if (size > PyUnicode_GET_SIZE(unicode))
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
392 size = PyUnicode_GET_SIZE(unicode) + 1;
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
393
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
394 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
395 memcpy(w, unicode->str, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
396 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
397 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
398 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
399 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
400 u = PyUnicode_AS_UNICODE(unicode);
33766
1907ad54d351 Applied patch for [ 1047269 ] Buffer overwrite in PyUnicode_AsWideChar.
Marc-André Lemburg <mal@egenix.com>
parents: 33499
diff changeset
401 for (i = size; i > 0; i--)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
402 *w++ = *u++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
403 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
404 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
405
34016
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
406 if (size > PyUnicode_GET_SIZE(unicode))
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
407 return PyUnicode_GET_SIZE(unicode);
264e45baacd5 Correct the handling of 0-termination of PyUnicode_AsWideChar()
Marc-André Lemburg <mal@egenix.com>
parents: 33766
diff changeset
408 else
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
409 return size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
410 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
411
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
412 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
413
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
414 PyObject *PyUnicode_FromOrdinal(int ordinal)
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
415 {
31873
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
416 Py_UNICODE s[1];
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
417
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
418 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
419 if (ordinal < 0 || ordinal > 0x10ffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
420 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
421 "unichr() arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
422 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
423 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
424 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
425 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
426 if (ordinal < 0 || ordinal > 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
427 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
428 "unichr() arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
429 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
430 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
431 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
432 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
433
31873
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
434 s[0] = (Py_UNICODE)ordinal;
227cb7402c6e SF Patch #926375: Remove a useless UTF-16 support code that is never
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31455
diff changeset
435 return PyUnicode_FromUnicode(s, 1);
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
436 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
437
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
438 PyObject *PyUnicode_FromObject(register PyObject *obj)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
439 {
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
440 /* XXX Perhaps we should make this API an alias of
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
441 PyObject_Unicode() instead ?! */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
442 if (PyUnicode_CheckExact(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
443 Py_INCREF(obj);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
444 return obj;
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
445 }
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
446 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
447 /* For a Unicode subtype that's not a Unicode object,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
448 return a true Unicode object with the same data. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
449 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(obj),
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
450 PyUnicode_GET_SIZE(obj));
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
451 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
452 return PyUnicode_FromEncodedObject(obj, NULL, "strict");
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
453 }
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
454
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
455 PyObject *PyUnicode_FromEncodedObject(register PyObject *obj,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
456 const char *encoding,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
457 const char *errors)
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
458 {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
459 const char *s = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
460 int len;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
461 PyObject *v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
462
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
463 if (obj == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
464 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
465 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
466 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
467
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
468 #if 0
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
469 /* For b/w compatibility we also accept Unicode objects provided
20563
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
470 that no encodings is given and then redirect to
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
471 PyObject_Unicode() which then applies the additional logic for
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
472 Unicode subclasses.
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
473
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
474 NOTE: This API should really only be used for object which
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
475 represent *encoded* Unicode !
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
476
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
477 */
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
478 if (PyUnicode_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
479 if (encoding) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
480 PyErr_SetString(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
481 "decoding Unicode is not supported");
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
482 return NULL;
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
483 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
484 return PyObject_Unicode(obj);
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
485 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
486 #else
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
487 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
488 PyErr_SetString(PyExc_TypeError,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
489 "decoding Unicode is not supported");
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
490 return NULL;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
491 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
492 #endif
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
493
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
494 /* Coerce object */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
495 if (PyString_Check(obj)) {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
496 s = PyString_AS_STRING(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
497 len = PyString_GET_SIZE(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
498 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
499 else if (PyObject_AsCharBuffer(obj, &s, &len)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
500 /* Overwrite the error message with something more useful in
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
501 case of a TypeError. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
502 if (PyErr_ExceptionMatches(PyExc_TypeError))
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
503 PyErr_Format(PyExc_TypeError,
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
504 "coercing to Unicode: need string or buffer, "
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
505 "%.80s found",
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
506 obj->ob_type->tp_name);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
507 goto onError;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
508 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
509
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
510 /* Convert to Unicode */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
511 if (len == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
512 Py_INCREF(unicode_empty);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
513 v = (PyObject *)unicode_empty;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
514 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
515 else
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
516 v = PyUnicode_Decode(s, len, encoding, errors);
15952
1f92d63b20f0 This patch adds a new builtin unistr() which behaves like str()
Marc-André Lemburg <mal@egenix.com>
parents: 15910
diff changeset
517
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
518 return v;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
519
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
520 onError:
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
521 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
522 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
523
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
524 PyObject *PyUnicode_Decode(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
525 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
526 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
527 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
528 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
529 PyObject *buffer = NULL, *unicode;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
530
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
531 if (encoding == NULL)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
532 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
533
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
534 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
535 if (strcmp(encoding, "utf-8") == 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
536 return PyUnicode_DecodeUTF8(s, size, errors);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
537 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
538 return PyUnicode_DecodeLatin1(s, size, errors);
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
539 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
540 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
541 return PyUnicode_DecodeMBCS(s, size, errors);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
542 #endif
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
543 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
544 return PyUnicode_DecodeASCII(s, size, errors);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
545
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
546 /* Decode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
547 buffer = PyBuffer_FromMemory((void *)s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
548 if (buffer == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
549 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
550 unicode = PyCodec_Decode(buffer, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
551 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
552 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
553 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
554 PyErr_Format(PyExc_TypeError,
12320
5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format
Guido van Rossum <guido@python.org>
parents: 12275
diff changeset
555 "decoder did not return an unicode object (type=%.400s)",
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
556 unicode->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
557 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
558 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
559 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
560 Py_DECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
561 return unicode;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
562
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
563 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
564 Py_XDECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
565 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
566 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
567
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
568 PyObject *PyUnicode_AsDecodedObject(PyObject *unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
569 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
570 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
571 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
572 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
573
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
574 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
575 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
576 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
577 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
578
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
579 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
580 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
581
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
582 /* Decode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
583 v = PyCodec_Decode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
584 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
585 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
586 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
587
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
588 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
589 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
590 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
591
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
592 PyObject *PyUnicode_Encode(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
593 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
594 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
595 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
596 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
597 PyObject *v, *unicode;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
598
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
599 unicode = PyUnicode_FromUnicode(s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
600 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
601 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
602 v = PyUnicode_AsEncodedString(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
603 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
604 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
605 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
606
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
607 PyObject *PyUnicode_AsEncodedObject(PyObject *unicode,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
608 const char *encoding,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
609 const char *errors)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
610 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
611 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
612
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
613 if (!PyUnicode_Check(unicode)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
614 PyErr_BadArgument();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
615 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
616 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
617
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
618 if (encoding == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
619 encoding = PyUnicode_GetDefaultEncoding();
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
620
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
621 /* Encode via the codec registry */
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
622 v = PyCodec_Encode(unicode, encoding, errors);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
623 if (v == NULL)
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
624 goto onError;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
625 return v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
626
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
627 onError:
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
628 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
629 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
630
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
631 PyObject *PyUnicode_AsEncodedString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
632 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
633 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
634 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
635 PyObject *v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
636
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
637 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
638 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
639 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
640 }
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
641
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
642 if (encoding == NULL)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
643 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
644
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
645 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
646 if (errors == NULL) {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
647 if (strcmp(encoding, "utf-8") == 0)
17995
837825a779c1 fix bogus indentation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17887
diff changeset
648 return PyUnicode_AsUTF8String(unicode);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
649 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
650 return PyUnicode_AsLatin1String(unicode);
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
651 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
652 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
653 return PyUnicode_AsMBCSString(unicode);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
654 #endif
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
655 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
656 return PyUnicode_AsASCIIString(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
657 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
658
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
659 /* Encode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
660 v = PyCodec_Encode(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
661 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
662 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
663 if (!PyString_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
664 PyErr_Format(PyExc_TypeError,
12320
5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format
Guido van Rossum <guido@python.org>
parents: 12275
diff changeset
665 "encoder did not return a string object (type=%.400s)",
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
666 v->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
667 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
668 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
669 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
670 return v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
671
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
672 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
673 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
674 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
675
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
676 PyObject *_PyUnicode_AsDefaultEncodedString(PyObject *unicode,
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
677 const char *errors)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
678 {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
679 PyObject *v = ((PyUnicodeObject *)unicode)->defenc;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
680
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
681 if (v)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
682 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
683 v = PyUnicode_AsEncodedString(unicode, NULL, errors);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
684 if (v && errors == NULL)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
685 ((PyUnicodeObject *)unicode)->defenc = v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
686 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
687 }
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
688
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
689 Py_UNICODE *PyUnicode_AsUnicode(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
690 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
691 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
692 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
693 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
694 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
695 return PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
696
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
697 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
698 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
699 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
700
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
701 int PyUnicode_GetSize(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
702 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
703 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
704 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
705 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
706 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
707 return PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
708
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
709 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
710 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
711 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
712
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
713 const char *PyUnicode_GetDefaultEncoding(void)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
714 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
715 return unicode_default_encoding;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
716 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
717
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
718 int PyUnicode_SetDefaultEncoding(const char *encoding)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
719 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
720 PyObject *v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
721
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
722 /* Make sure the encoding is valid. As side effect, this also
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
723 loads the encoding into the codec registry cache. */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
724 v = _PyCodec_Lookup(encoding);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
725 if (v == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
726 goto onError;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
727 Py_DECREF(v);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
728 strncpy(unicode_default_encoding,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
729 encoding,
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
730 sizeof(unicode_default_encoding));
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
731 return 0;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
732
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
733 onError:
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
734 return -1;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
735 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
736
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
737 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
738 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
739 if no exception occured, copy the replacement to the output
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
740 and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
741 return 0 on success, -1 on error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
742 */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
743
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
744 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
745 int unicode_decode_call_errorhandler(const char *errors, PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
746 const char *encoding, const char *reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
747 const char *input, int insize, int *startinpos, int *endinpos, PyObject **exceptionObject, const char **inptr,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
748 PyObject **output, int *outpos, Py_UNICODE **outptr)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
749 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
750 static char *argparse = "O!i;decoding error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
751
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
752 PyObject *restuple = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
753 PyObject *repunicode = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
754 int outsize = PyUnicode_GET_SIZE(*output);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
755 int requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
756 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
757 Py_UNICODE *repptr;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
758 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
759 int res = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
760
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
761 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
762 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
763 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
764 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
765 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
766
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
767 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
768 *exceptionObject = PyUnicodeDecodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
769 encoding, input, insize, *startinpos, *endinpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
770 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
771 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
772 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
773 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
774 if (PyUnicodeDecodeError_SetStart(*exceptionObject, *startinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
775 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
776 if (PyUnicodeDecodeError_SetEnd(*exceptionObject, *endinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
777 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
778 if (PyUnicodeDecodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
779 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
780 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
781
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
782 restuple = PyObject_CallFunctionObjArgs(*errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
783 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
784 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
785 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
786 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
787 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
788 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
789 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type, &repunicode, &newpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
790 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
791 if (newpos<0)
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
792 newpos = insize+newpos;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
793 if (newpos<0 || newpos>insize) {
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
794 PyErr_Format(PyExc_IndexError, "position %d from error handler out of bounds", newpos);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
795 goto onError;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
796 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
797
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
798 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
799 have+the replacement+the rest of the string (starting
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
800 at the new input position), so we won't have to check space
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
801 when there are no errors in the rest of the string) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
802 repptr = PyUnicode_AS_UNICODE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
803 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
804 requiredsize = *outpos + repsize + insize-newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
805 if (requiredsize > outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
806 if (requiredsize<2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
807 requiredsize = 2*outsize;
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
808 if (PyUnicode_Resize(output, requiredsize) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
809 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
810 *outptr = PyUnicode_AS_UNICODE(*output) + *outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
811 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
812 *endinpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
813 *inptr = input + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
814 Py_UNICODE_COPY(*outptr, repptr, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
815 *outptr += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
816 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
817 /* we made it! */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
818 res = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
819
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
820 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
821 Py_XDECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
822 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
823 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
824
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
825 /* --- UTF-7 Codec -------------------------------------------------------- */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
826
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
827 /* see RFC2152 for details */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
828
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
829 static
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
830 char utf7_special[128] = {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
831 /* indicate whether a UTF-7 character is special i.e. cannot be directly
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
832 encoded:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
833 0 - not special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
834 1 - special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
835 2 - whitespace (optional)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
836 3 - RFC2152 Set O (optional) */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
837 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
838 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
839 2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
840 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
841 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
842 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
843 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
844 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
845
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
846 };
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
847
39907
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
848 /* Note: The comparison (c) <= 0 is a trick to work-around gcc
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
849 warnings about the comparison always being false; since
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
850 utf7_special[0] is 1, we can safely make that one comparison
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
851 true */
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
852
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
853 #define SPECIAL(c, encodeO, encodeWS) \
39907
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
854 ((c) > 127 || (c) <= 0 || utf7_special[(c)] == 1 || \
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
855 (encodeWS && (utf7_special[(c)] == 2)) || \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
856 (encodeO && (utf7_special[(c)] == 3)))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
857
39907
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
858 #define B64(n) \
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
859 ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[(n) & 0x3f])
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
860 #define B64CHAR(c) \
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
861 (isalnum(c) || (c) == '+' || (c) == '/')
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
862 #define UB64(c) \
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
863 ((c) == '+' ? 62 : (c) == '/' ? 63 : (c) >= 'a' ? \
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
864 (c) - 71 : (c) >= 'A' ? (c) - 65 : (c) + 4 )
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
865
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
866 #define ENCODE(out, ch, bits) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
867 while (bits >= 6) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
868 *out++ = B64(ch >> (bits-6)); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
869 bits -= 6; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
870 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
871
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
872 #define DECODE(out, ch, bits, surrogate) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
873 while (bits >= 16) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
874 Py_UNICODE outCh = (Py_UNICODE) ((ch >> (bits-16)) & 0xffff); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
875 bits -= 16; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
876 if (surrogate) { \
39907
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
877 /* We have already generated an error for the high surrogate \
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
878 so let's not bother seeing if the low surrogate is correct or not */ \
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
879 surrogate = 0; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
880 } else if (0xDC00 <= outCh && outCh <= 0xDFFF) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
881 /* This is a surrogate pair. Unfortunately we can't represent \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
882 it in a 16-bit character */ \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
883 surrogate = 1; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
884 errmsg = "code pairs are not supported"; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
885 goto utf7Error; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
886 } else { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
887 *out++ = outCh; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
888 } \
39907
2632c9af64ab [Backport rev. 39743 by lemburg]
Andrew M. Kuchling <amk@amk.ca>
parents: 39630
diff changeset
889 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
890
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
891 PyObject *PyUnicode_DecodeUTF7(const char *s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
892 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
893 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
894 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
895 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
896 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
897 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
898 int outpos;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
899 const char *e;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
900 PyUnicodeObject *unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
901 Py_UNICODE *p;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
902 const char *errmsg = "";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
903 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
904 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
905 unsigned long charsleft = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
906 int surrogate = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
907 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
908 PyObject *exc = NULL;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
909
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
910 unicode = _PyUnicode_New(size);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
911 if (!unicode)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
912 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
913 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
914 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
915
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
916 p = unicode->str;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
917 e = s + size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
918
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
919 while (s < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
920 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
921 restart:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
922 ch = *s;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
923
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
924 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
925 if ((ch == '-') || !B64CHAR(ch)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
926 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
927 s++;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
928
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
929 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
930 if (bitsleft >= 6) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
931 /* The shift sequence has a partial character in it. If
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
932 bitsleft < 6 then we could just classify it as padding
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
933 but that is not the case here */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
934
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
935 errmsg = "partial character in shift sequence";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
936 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
937 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
938 /* According to RFC2152 the remaining bits should be zero. We
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
939 choose to signal an error/insert a replacement character
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
940 here so indicate the potential of a misencoded character. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
941
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
942 /* On x86, a << b == a << (b%32) so make sure that bitsleft != 0 */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
943 if (bitsleft && charsleft << (sizeof(charsleft) * 8 - bitsleft)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
944 errmsg = "non-zero padding bits in shift sequence";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
945 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
946 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
947
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
948 if (ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
949 if ((s < e) && (*(s) == '-')) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
950 *p++ = '-';
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
951 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
952 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
953 } else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
954 errmsg = "unexpected special character";
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
955 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
956 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
957 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
958 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
959 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
960 charsleft = (charsleft << 6) | UB64(ch);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
961 bitsleft += 6;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
962 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
963 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
964 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
965 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
966 else if ( ch == '+' ) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
967 startinpos = s-starts;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
968 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
969 if (s < e && *s == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
970 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
971 *p++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
972 } else
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
973 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
974 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
975 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
976 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
977 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
978 else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
979 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
980 s++;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
981 goto utf7Error;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
982 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
983 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
984 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
985 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
986 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
987 continue;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
988 utf7Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
989 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
990 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
991 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
992 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
993 "utf7", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
994 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
995 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
996 goto onError;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
997 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
999 if (inShift) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1000 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1001 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1002 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1003 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1004 "utf7", "unterminated shift sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1005 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1006 (PyObject **)&unicode, &outpos, &p))
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1007 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1008 if (s < e)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1009 goto restart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1010 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1011
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1012 if (_PyUnicode_Resize(&unicode, p - PyUnicode_AS_UNICODE(unicode)) < 0)
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1013 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1014
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1015 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1016 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1017 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1018
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1019 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1020 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1021 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1022 Py_DECREF(unicode);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1023 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1024 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1025
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1026
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1027 PyObject *PyUnicode_EncodeUTF7(const Py_UNICODE *s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1028 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1029 int encodeSetO,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1030 int encodeWhiteSpace,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1031 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1032 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1033 PyObject *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1034 /* It might be possible to tighten this worst case */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1035 unsigned int cbAllocated = 5 * size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1036 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1037 int i = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1038 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1039 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1040 char * out;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1041 char * start;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1042
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1043 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1044 return PyString_FromStringAndSize(NULL, 0);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1045
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1046 v = PyString_FromStringAndSize(NULL, cbAllocated);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1047 if (v == NULL)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1048 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1049
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1050 start = out = PyString_AS_STRING(v);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1051 for (;i < size; ++i) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1052 Py_UNICODE ch = s[i];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1053
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1054 if (!inShift) {
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1055 if (ch == '+') {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1056 *out++ = '+';
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1057 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1058 } else if (SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1059 charsleft = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1060 bitsleft = 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1061 *out++ = '+';
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1062 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1063 inShift = bitsleft > 0;
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1064 } else {
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1065 *out++ = (char) ch;
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1066 }
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1067 } else {
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1068 if (!SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1069 *out++ = B64(charsleft << (6-bitsleft));
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1070 charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1071 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1072 /* Characters not in the BASE64 set implicitly unshift the sequence
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1073 so no '-' is required, except if the character is itself a '-' */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1074 if (B64CHAR(ch) || ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1075 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1076 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1077 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1078 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1079 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1080 bitsleft += 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1081 charsleft = (charsleft << 16) | ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1082 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1083
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1084 /* If the next character is special then we dont' need to terminate
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1085 the shift sequence. If the next character is not a BASE64 character
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1086 or '-' then the shift sequence will be terminated implicitly and we
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1087 don't have to insert a '-'. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1088
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1089 if (bitsleft == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1090 if (i + 1 < size) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1091 Py_UNICODE ch2 = s[i+1];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1092
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1093 if (SPECIAL(ch2, encodeSetO, encodeWhiteSpace)) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1094
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1095 } else if (B64CHAR(ch2) || ch2 == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1096 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1097 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1098 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1099 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1100 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1101
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1102 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1103 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1104 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1105 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1106 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1107 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1108 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1109 }
31325
b4532c246a3a Cosmetic fix for wrongly indented tabs with ts=4.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31252
diff changeset
1110 }
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1111 if (bitsleft) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1112 *out++= B64(charsleft << (6-bitsleft) );
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1113 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1114 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1115
23295
6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people
Tim Peters <tim.peters@gmail.com>
parents: 23294
diff changeset
1116 _PyString_Resize(&v, out - start);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1117 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1118 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1119
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1120 #undef SPECIAL
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1121 #undef B64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1122 #undef B64CHAR
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1123 #undef UB64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1124 #undef ENCODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1125 #undef DECODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1126
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1127 /* --- UTF-8 Codec -------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1128
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1129 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1130 char utf8_code_length[256] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1131 /* Map UTF-8 encoded prefix byte to sequence length. zero means
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1132 illegal prefix. see RFC 2279 for details */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1133 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1134 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1135 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1136 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1137 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1138 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1139 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1140 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1141 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1142 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1143 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1144 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1145 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1146 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1147 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1148 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1149 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1150
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1151 PyObject *PyUnicode_DecodeUTF8(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1152 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1153 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1154 {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1155 return PyUnicode_DecodeUTF8Stateful(s, size, errors, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1156 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1157
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1158 PyObject *PyUnicode_DecodeUTF8Stateful(const char *s,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1159 int size,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1160 const char *errors,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1161 int *consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1162 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1163 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1164 int n;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1165 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1166 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1167 int outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1168 const char *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1169 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1170 Py_UNICODE *p;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1171 const char *errmsg = "";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1172 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1173 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1174
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1175 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1176 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1177 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1178 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1179 return NULL;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1180 if (size == 0) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1181 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1182 *consumed = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1183 return (PyObject *)unicode;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1184 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1185
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1186 /* Unpack UTF-8 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1187 p = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1188 e = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1189
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1190 while (s < e) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1191 Py_UCS4 ch = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1192
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1193 if (ch < 0x80) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1194 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1195 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1196 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1197 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1198
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1199 n = utf8_code_length[ch];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1200
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1201 if (s + n > e) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1202 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1203 break;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1204 else {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1205 errmsg = "unexpected end of data";
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1206 startinpos = s-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1207 endinpos = size;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1208 goto utf8Error;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1209 }
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1210 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1211
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1212 switch (n) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1213
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1214 case 0:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1215 errmsg = "unexpected code byte";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1216 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1217 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1218 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1219
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1220 case 1:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1221 errmsg = "internal error";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1222 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1223 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1224 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1226 case 2:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1227 if ((s[1] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1228 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1229 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1230 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1231 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1232 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1233 ch = ((s[0] & 0x1f) << 6) + (s[1] & 0x3f);
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1234 if (ch < 0x80) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1235 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1236 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1237 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1238 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1239 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1240 else
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1241 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1242 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1243
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1244 case 3:
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1245 if ((s[1] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1246 (s[2] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1247 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1248 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1249 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1250 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1251 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1252 ch = ((s[0] & 0x0f) << 12) + ((s[1] & 0x3f) << 6) + (s[2] & 0x3f);
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1253 if (ch < 0x0800) {
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1254 /* Note: UTF-8 encodings of surrogates are considered
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1255 legal UTF-8 sequences;
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1256
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1257 XXX For wide builds (UCS-4) we should probably try
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1258 to recombine the surrogates into a single code
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1259 unit.
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1260 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1261 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1262 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1263 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1264 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1265 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1266 else
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1267 *p++ = (Py_UNICODE)ch;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1268 break;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1269
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1270 case 4:
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1271 if ((s[1] & 0xc0) != 0x80 ||
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1272 (s[2] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1273 (s[3] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1274 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1275 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1276 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1277 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1278 }
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1279 ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1280 ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1281 /* validate and convert to UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1282 if ((ch < 0x10000) /* minimum value allowed for 4
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1283 byte encoding */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1284 || (ch > 0x10ffff)) /* maximum value allowed for
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1285 UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1286 {
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1287 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1288 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1289 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1290 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1291 }
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1292 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1293 *p++ = (Py_UNICODE)ch;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1294 #else
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1295 /* compute and append the two surrogates: */
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1296
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1297 /* translate from 10000..10FFFF to 0..FFFF */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1298 ch -= 0x10000;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1299
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1300 /* high surrogate = top 10 bits added to D800 */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1301 *p++ = (Py_UNICODE)(0xD800 + (ch >> 10));
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1302
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1303 /* low surrogate = bottom 10 bits added to DC00 */
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
1304 *p++ = (Py_UNICODE)(0xDC00 + (ch & 0x03FF));
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1305 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1306 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1307
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1308 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1309 /* Other sizes are only needed for UCS-4 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1310 errmsg = "unsupported Unicode code range";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1311 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1312 endinpos = startinpos+n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1313 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1314 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1315 s += n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1316 continue;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1317
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1318 utf8Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1319 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1320 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1321 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1322 "utf8", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1323 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1324 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1325 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1326 }
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1327 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1328 *consumed = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1329
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1330 /* Adjust length */
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1331 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1332 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1333
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1334 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1335 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1336 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1337
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1338 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1339 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1340 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1341 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1342 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1343 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1344
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1345 /* Allocation strategy: if the string is short, convert into a stack buffer
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1346 and allocate exactly as much space needed at the end. Else allocate the
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1347 maximum possible needed (4 result bytes per Unicode character), and return
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1348 the excess memory at the end.
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1349 */
23198
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1350 PyObject *
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1351 PyUnicode_EncodeUTF8(const Py_UNICODE *s,
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1352 int size,
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1353 const char *errors)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1354 {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1355 #define MAX_SHORT_UNICHARS 300 /* largest size we'll do on the stack */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1356
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1357 int i; /* index into s of next input byte */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1358 PyObject *v; /* result string object */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1359 char *p; /* next free byte in output buffer */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1360 int nallocated; /* number of result bytes allocated */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1361 int nneeded; /* number of result bytes needed */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1362 char stackbuf[MAX_SHORT_UNICHARS * 4];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1363
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1364 assert(s != NULL);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1365 assert(size >= 0);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1366
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1367 if (size <= MAX_SHORT_UNICHARS) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1368 /* Write into the stack buffer; nallocated can't overflow.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1369 * At the end, we'll allocate exactly as much heap space as it
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1370 * turns out we need.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1371 */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1372 nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1373 v = NULL; /* will allocate after we're done */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1374 p = stackbuf;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1375 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1376 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1377 /* Overallocate on the heap, and give the excess back at the end. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1378 nallocated = size * 4;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1379 if (nallocated / 4 != size) /* overflow! */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1380 return PyErr_NoMemory();
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1381 v = PyString_FromStringAndSize(NULL, nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1382 if (v == NULL)
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1383 return NULL;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1384 p = PyString_AS_STRING(v);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1385 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1386
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1387 for (i = 0; i < size;) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1388 Py_UCS4 ch = s[i++];
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1389
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1390 if (ch < 0x80)
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1391 /* Encode ASCII */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1392 *p++ = (char) ch;
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1393
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1394 else if (ch < 0x0800) {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1395 /* Encode Latin-1 */
21884
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1396 *p++ = (char)(0xc0 | (ch >> 6));
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1397 *p++ = (char)(0x80 | (ch & 0x3f));
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1398 }
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1399 else {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1400 /* Encode UCS2 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1401 if (ch < 0x10000) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1402 /* Special case: check for high surrogate */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1403 if (0xD800 <= ch && ch <= 0xDBFF && i != size) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1404 Py_UCS4 ch2 = s[i];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1405 /* Check for low surrogate and combine the two to
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1406 form a UCS4 value */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1407 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1408 ch = ((ch - 0xD800) << 10 | (ch2 - 0xDC00)) + 0x10000;
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1409 i++;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1410 goto encodeUCS4;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1411 }
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1412 /* Fall through: handles isolated high surrogates */
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1413 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1414 *p++ = (char)(0xe0 | (ch >> 12));
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1415 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1416 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1417 continue;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1418 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1419 encodeUCS4:
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1420 /* Encode UCS4 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1421 *p++ = (char)(0xf0 | (ch >> 18));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1422 *p++ = (char)(0x80 | ((ch >> 12) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1423 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1424 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1425 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1426 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1427
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1428 if (v == NULL) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1429 /* This was stack allocated. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1430 nneeded = Py_SAFE_DOWNCAST(p - stackbuf, long, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1431 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1432 v = PyString_FromStringAndSize(stackbuf, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1433 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1434 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1435 /* Cut back to size actually needed. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1436 nneeded = Py_SAFE_DOWNCAST(p - PyString_AS_STRING(v), long, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1437 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1438 _PyString_Resize(&v, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1439 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1440 return v;
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1441
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1442 #undef MAX_SHORT_UNICHARS
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1443 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1444
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1445 PyObject *PyUnicode_AsUTF8String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1446 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1447 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1448 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1449 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1450 }
14059
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1451 return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1452 PyUnicode_GET_SIZE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1453 NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1454 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1455
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1456 /* --- UTF-16 Codec ------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1457
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1458 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1459 PyUnicode_DecodeUTF16(const char *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1460 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1461 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1462 int *byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1463 {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1464 return PyUnicode_DecodeUTF16Stateful(s, size, errors, byteorder, NULL);
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1465 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1466
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1467 PyObject *
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1468 PyUnicode_DecodeUTF16Stateful(const char *s,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1469 int size,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1470 const char *errors,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1471 int *byteorder,
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1472 int *consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1473 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1474 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1475 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1476 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1477 int outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1478 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1479 Py_UNICODE *p;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1480 const unsigned char *q, *e;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1481 int bo = 0; /* assume native ordering by default */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1482 const char *errmsg = "";
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1483 /* Offsets from q for retrieving byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1484 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1485 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1486 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1487 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1488 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1489 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1490 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1491
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1492 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1493 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1494 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1495 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1496 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1497 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1498 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1499
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1500 /* Unpack UTF-16 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1501 p = unicode->str;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1502 q = (unsigned char *)s;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1503 e = q + size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1504
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1505 if (byteorder)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1506 bo = *byteorder;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1507
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1508 /* Check for BOM marks (U+FEFF) in the input and adjust current
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1509 byte order setting accordingly. In native mode, the leading BOM
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1510 mark is skipped, in all other modes, it is copied to the output
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1511 stream as-is (giving a ZWNBSP character). */
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1512 if (bo == 0) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1513 if (size >= 2) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1514 const Py_UNICODE bom = (q[ihi] << 8) | q[ilo];
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1515 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1516 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1517 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1518 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1519 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1520 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1521 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1522 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1523 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1524 #else
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1525 if (bom == 0xFEFF) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1526 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1527 bo = 1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1528 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1529 else if (bom == 0xFFFE) {
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1530 q += 2;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1531 bo = -1;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1532 }
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1533 #endif
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1534 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1535 }
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1536
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1537 if (bo == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1538 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1539 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1540 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1541 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1542 else if (bo == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1543 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1544 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1545 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1546 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1547
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1548 while (q < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1549 Py_UNICODE ch;
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1550 /* remaining bytes at the end? (size should be even) */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1551 if (e-q<2) {
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1552 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1553 break;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1554 errmsg = "truncated data";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1555 startinpos = ((const char *)q)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1556 endinpos = ((const char *)e)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1557 goto utf16Error;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1558 /* The remaining input chars are ignored if the callback
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1559 chooses to skip the input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1560 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1561 ch = (q[ihi] << 8) | q[ilo];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1562
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1563 q += 2;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1564
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1565 if (ch < 0xD800 || ch > 0xDFFF) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1566 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1567 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1568 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1569
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1570 /* UTF-16 code pair: */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1571 if (q >= e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1572 errmsg = "unexpected end of data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1573 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1574 endinpos = ((const char *)e)-starts;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1575 goto utf16Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1576 }
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
1577 if (0xD800 <= ch && ch <= 0xDBFF) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1578 Py_UNICODE ch2 = (q[ihi] << 8) | q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1579 q += 2;
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
1580 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1581 #ifndef Py_UNICODE_WIDE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1582 *p++ = ch;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1583 *p++ = ch2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1584 #else
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1585 *p++ = (((ch & 0x3FF)<<10) | (ch2 & 0x3FF)) + 0x10000;
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1586 #endif
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1587 continue;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1588 }
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1589 else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1590 errmsg = "illegal UTF-16 surrogate";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1591 startinpos = (((const char *)q)-4)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1592 endinpos = startinpos+2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1593 goto utf16Error;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1594 }
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1595
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1596 }
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1597 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1598 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1599 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1600 /* Fall through to report the error */
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1601
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1602 utf16Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1603 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1604 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1605 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1606 "utf16", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1607 starts, size, &startinpos, &endinpos, &exc, (const char **)&q,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1608 (PyObject **)&unicode, &outpos, &p))
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1609 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1610 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1611
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1612 if (byteorder)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1613 *byteorder = bo;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1614
33428
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1615 if (consumed)
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1616 *consumed = (const char *)q-starts;
57478a13fbf6 SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now support
Walter Dörwald <walter@livinglogic.de>
parents: 33311
diff changeset
1617
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1618 /* Adjust length */
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1619 if (_PyUnicode_Resize(&unicode, p - unicode->str) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1620 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1621
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1622 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1623 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1624 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1625
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1626 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1627 Py_DECREF(unicode);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1628 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1629 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1630 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1631 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1632
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1633 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1634 PyUnicode_EncodeUTF16(const Py_UNICODE *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1635 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1636 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1637 int byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1638 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1639 PyObject *v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1640 unsigned char *p;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1641 #ifdef Py_UNICODE_WIDE
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1642 int i, pairs;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1643 #else
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1644 const int pairs = 0;
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1645 #endif
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1646 /* Offsets from p for storing byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1647 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1648 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1649 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1650 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1651 #endif
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1652
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1653 #define STORECHAR(CH) \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1654 do { \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1655 p[ihi] = ((CH) >> 8) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1656 p[ilo] = (CH) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1657 p += 2; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1658 } while(0)
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1659
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1660 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1661 for (i = pairs = 0; i < size; i++)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1662 if (s[i] >= 0x10000)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1663 pairs++;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1664 #endif
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
1665 v = PyString_FromStringAndSize(NULL,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1666 2 * (size + pairs + (byteorder == 0)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1667 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1668 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1669
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1670 p = (unsigned char *)PyString_AS_STRING(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1671 if (byteorder == 0)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1672 STORECHAR(0xFEFF);
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
1673 if (size == 0)
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
1674 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1675
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1676 if (byteorder == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1677 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1678 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1679 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1680 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1681 else if (byteorder == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1682 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1683 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1684 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1685 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1686
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1687 while (size-- > 0) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1688 Py_UNICODE ch = *s++;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1689 Py_UNICODE ch2 = 0;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1690 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1691 if (ch >= 0x10000) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1692 ch2 = 0xDC00 | ((ch-0x10000) & 0x3FF);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1693 ch = 0xD800 | ((ch-0x10000) >> 10);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1694 }
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
1695 #endif
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1696 STORECHAR(ch);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1697 if (ch2)
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1698 STORECHAR(ch2);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1699 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1700 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1701 #undef STORECHAR
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1702 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1703
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1704 PyObject *PyUnicode_AsUTF16String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1705 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1706 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1707 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1708 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1709 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1710 return PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1711 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1712 NULL,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1713 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1714 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1715
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1716 /* --- Unicode Escape Codec ----------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1717
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1718 static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL;
13063
eb243443b884 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13049
diff changeset
1719
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1720 PyObject *PyUnicode_DecodeUnicodeEscape(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1721 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1722 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1723 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1724 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1725 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1726 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1727 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1728 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1729 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1730 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1731 const char *end;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1732 char* message;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1733 Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1734 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1735 PyObject *exc = NULL;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1736
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1737 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1738 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1739 length after conversion to the true value.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1740 (but if the error callback returns a long replacement string
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1741 we'll have to allocate more space) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1742 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1743 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1744 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1745 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1746 return (PyObject *)v;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1747
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1748 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1749 end = s + size;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1750
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1751 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1752 unsigned char c;
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
1753 Py_UNICODE x;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1754 int digits;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1755
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1756 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1757 if (*s != '\\') {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1758 *p++ = (unsigned char) *s++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1759 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1760 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1761
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1762 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1763 /* \ - Escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1764 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1765 switch (*s++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1766
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1767 /* \x escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1768 case '\n': break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1769 case '\\': *p++ = '\\'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1770 case '\'': *p++ = '\''; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1771 case '\"': *p++ = '\"'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1772 case 'b': *p++ = '\b'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1773 case 'f': *p++ = '\014'; break; /* FF */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1774 case 't': *p++ = '\t'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1775 case 'n': *p++ = '\n'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1776 case 'r': *p++ = '\r'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1777 case 'v': *p++ = '\013'; break; /* VT */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1778 case 'a': *p++ = '\007'; break; /* BEL, not classic C */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1779
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1780 /* \OOO (octal) escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1781 case '0': case '1': case '2': case '3':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1782 case '4': case '5': case '6': case '7':
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1783 x = s[-1] - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1784 if ('0' <= *s && *s <= '7') {
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1785 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1786 if ('0' <= *s && *s <= '7')
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1787 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1788 }
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1789 *p++ = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1790 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1791
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1792 /* hex escapes */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1793 /* \xXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1794 case 'x':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1795 digits = 2;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1796 message = "truncated \\xXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1797 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1798
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1799 /* \uXXXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1800 case 'u':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1801 digits = 4;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1802 message = "truncated \\uXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1803 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1804
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1805 /* \UXXXXXXXX */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1806 case 'U':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1807 digits = 8;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1808 message = "truncated \\UXXXXXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1809 hexescape:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1810 chr = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1811 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1812 if (s+digits>end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1813 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1814 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1815 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1816 "unicodeescape", "end of string in escape sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1817 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1818 (PyObject **)&v, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1819 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1820 goto nextByte;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1821 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1822 for (i = 0; i < digits; ++i) {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1823 c = (unsigned char) s[i];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1824 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1825 endinpos = (s+i+1)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1826 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1827 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1828 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1829 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1830 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1831 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1832 goto nextByte;
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1833 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1834 chr = (chr<<4) & ~0xF;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1835 if (c >= '0' && c <= '9')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1836 chr += c - '0';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1837 else if (c >= 'a' && c <= 'f')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1838 chr += 10 + c - 'a';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1839 else
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1840 chr += 10 + c - 'A';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1841 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1842 s += i;
30653
3b47d6057151 Fix for SF bug [ 817156 ] invalid \U escape gives 0=length unistr.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30441
diff changeset
1843 if (chr == 0xffffffff && PyErr_Occurred())
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1844 /* _decoding_error will have already written into the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1845 target buffer. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1846 break;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1847 store:
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1848 /* when we get here, chr is a 32-bit unicode character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1849 if (chr <= 0xffff)
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1850 /* UCS-2 character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1851 *p++ = (Py_UNICODE) chr;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1852 else if (chr <= 0x10ffff) {
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1853 /* UCS-4 character. Either store directly, or as
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1854 surrogate pair. */
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1855 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1856 *p++ = chr;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1857 #else
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1858 chr -= 0x10000L;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1859 *p++ = 0xD800 + (Py_UNICODE) (chr >> 10);
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
1860 *p++ = 0xDC00 + (Py_UNICODE) (chr & 0x03FF);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1861 #endif
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1862 } else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1863 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1864 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1865 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1866 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1867 "unicodeescape", "illegal Unicode character",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1868 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1869 (PyObject **)&v, &outpos, &p))
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1870 goto onError;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1871 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1872 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1873
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1874 /* \N{name} */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1875 case 'N':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1876 message = "malformed \\N character escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1877 if (ucnhash_CAPI == NULL) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1878 /* load the unicode data module */
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
1879 PyObject *m, *api;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1880 m = PyImport_ImportModule("unicodedata");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1881 if (m == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1882 goto ucnhashError;
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
1883 api = PyObject_GetAttrString(m, "ucnhash_CAPI");
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1884 Py_DECREF(m);
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
1885 if (api == NULL)
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1886 goto ucnhashError;
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
1887 ucnhash_CAPI = PyCObject_AsVoidPtr(api);
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
1888 Py_DECREF(api);
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1889 if (ucnhash_CAPI == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1890 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1891 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1892 if (*s == '{') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1893 const char *start = s+1;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1894 /* look for the closing brace */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1895 while (*s != '}' && s < end)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1896 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1897 if (s > start && s < end && *s == '}') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1898 /* found a name. look it up in the unicode database */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1899 message = "unknown Unicode character name";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1900 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1901 if (ucnhash_CAPI->getcode(start, s-start-1, &chr))
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1902 goto store;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1903 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1904 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1905 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1906 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1907 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1908 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1909 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1910 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1911 (PyObject **)&v, &outpos, &p))
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1912 goto onError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1913 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1914
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1915 default:
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1916 if (s > end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1917 message = "\\ at end of string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1918 s--;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1919 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1920 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1921 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1922 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1923 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1924 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1925 (PyObject **)&v, &outpos, &p))
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1926 goto onError;
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1927 }
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1928 else {
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1929 *p++ = '\\';
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1930 *p++ = (unsigned char)s[-1];
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1931 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1932 break;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1933 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1934 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1935 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1936 }
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
1937 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1938 goto onError;
30160
3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()).
Walter Dörwald <walter@livinglogic.de>
parents: 30149
diff changeset
1939 Py_XDECREF(errorHandler);
3329b058db20 Fix another refcounting leak (in PyUnicode_DecodeUnicodeEscape()).
Walter Dörwald <walter@livinglogic.de>
parents: 30149
diff changeset
1940 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1941 return (PyObject *)v;
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1942
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1943 ucnhashError:
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1944 PyErr_SetString(
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1945 PyExc_UnicodeError,
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1946 "\\N escapes not supported (can't load unicodedata module)"
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1947 );
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
1948 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1949 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1950 Py_XDECREF(exc);
16064
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
1951 return NULL;
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
1952
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1953 onError:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1954 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1955 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1956 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1957 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1958 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1959
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1960 /* Return a Unicode-Escape string version of the Unicode object.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1961
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1962 If quotes is true, the string is enclosed in u"" or u'' quotes as
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1963 appropriate.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1964
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1965 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1966
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1967 static const Py_UNICODE *findchar(const Py_UNICODE *s,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1968 int size,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1969 Py_UNICODE ch);
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1970
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1971 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1972 PyObject *unicodeescape_string(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1973 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1974 int quotes)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1975 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1976 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1977 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1978
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1979 static const char *hexdigit = "0123456789abcdef";
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1980
39630
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1981 /* Initial allocation is based on the longest-possible unichr
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1982 escape.
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1983
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1984 In wide (UTF-32) builds '\U00xxxxxx' is 10 chars per source
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1985 unichr, so in this case it's the longest unichr escape. In
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1986 narrow (UTF-16) builds this is five chars per source unichr
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1987 since there are two unichrs in the surrogate pair, so in narrow
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1988 (UTF-16) builds it's not the longest unichr escape.
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1989
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1990 In wide or narrow builds '\uxxxx' is 6 chars per source unichr,
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1991 so in the narrow (UTF-16) build case it's the longest unichr
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1992 escape.
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1993 */
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1994
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1995 repr = PyString_FromStringAndSize(NULL,
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1996 2
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1997 #ifdef Py_UNICODE_WIDE
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1998 + 10*size
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
1999 #else
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
2000 + 6*size
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
2001 #endif
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
2002 + 1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2003 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2004 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2005
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2006 p = PyString_AS_STRING(repr);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2007
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2008 if (quotes) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2009 *p++ = 'u';
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2010 *p++ = (findchar(s, size, '\'') &&
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2011 !findchar(s, size, '"')) ? '"' : '\'';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2012 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2013 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2014 Py_UNICODE ch = *s++;
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2015
37267
d8de7500291d after discussions with perky, reverted fix for Bug #1379994: Builtin
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37261
diff changeset
2016 /* Escape quotes */
d8de7500291d after discussions with perky, reverted fix for Bug #1379994: Builtin
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37261
diff changeset
2017 if (quotes &&
d8de7500291d after discussions with perky, reverted fix for Bug #1379994: Builtin
Anthony Baxter <anthonybaxter@gmail.com>
parents: 37261
diff changeset
2018 (ch == (Py_UNICODE) PyString_AS_STRING(repr)[1] || ch == '\\')) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2019 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2020 *p++ = (char) ch;
20018
8aa1ada33f64 Fix a bug in rendering of \\ by repr() -- it rendered as \\\ instead
Guido van Rossum <guido@python.org>
parents: 20007
diff changeset
2021 continue;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2022 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2023
18773
743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This
Guido van Rossum <guido@python.org>
parents: 18324
diff changeset
2024 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2025 /* Map 21-bit characters to '\U00xxxxxx' */
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2026 else if (ch >= 0x10000) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2027 *p++ = '\\';
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2028 *p++ = 'U';
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2029 *p++ = hexdigit[(ch >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2030 *p++ = hexdigit[(ch >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2031 *p++ = hexdigit[(ch >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2032 *p++ = hexdigit[(ch >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2033 *p++ = hexdigit[(ch >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2034 *p++ = hexdigit[(ch >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2035 *p++ = hexdigit[(ch >> 4) & 0x0000000F];
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2036 *p++ = hexdigit[ch & 0x0000000F];
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2037 continue;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
2038 }
39630
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
2039 #else
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
2040 /* Map UTF-16 surrogate pairs to '\U00xxxxxx' */
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2041 else if (ch >= 0xD800 && ch < 0xDC00) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2042 Py_UNICODE ch2;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2043 Py_UCS4 ucs;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2044
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2045 ch2 = *s++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2046 size--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2047 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2048 ucs = (((ch & 0x03FF) << 10) | (ch2 & 0x03FF)) + 0x00010000;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2049 *p++ = '\\';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2050 *p++ = 'U';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2051 *p++ = hexdigit[(ucs >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2052 *p++ = hexdigit[(ucs >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2053 *p++ = hexdigit[(ucs >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2054 *p++ = hexdigit[(ucs >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2055 *p++ = hexdigit[(ucs >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2056 *p++ = hexdigit[(ucs >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2057 *p++ = hexdigit[(ucs >> 4) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2058 *p++ = hexdigit[ucs & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2059 continue;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2060 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2061 /* Fall through: isolated surrogates are copied as-is */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2062 s--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2063 size++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2064 }
39630
2f6b42aead94 Backport rev 51448:
Georg Brandl <georg@python.org>
parents: 38606
diff changeset
2065 #endif
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2066
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2067 /* Map 16-bit characters to '\uxxxx' */
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2068 if (ch >= 256) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2069 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2070 *p++ = 'u';
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2071 *p++ = hexdigit[(ch >> 12) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2072 *p++ = hexdigit[(ch >> 8) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2073 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2074 *p++ = hexdigit[ch & 0x000F];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2075 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2076
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2077 /* Map special whitespace to '\t', \n', '\r' */
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2078 else if (ch == '\t') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2079 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2080 *p++ = 't';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2081 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2082 else if (ch == '\n') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2083 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2084 *p++ = 'n';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2085 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2086 else if (ch == '\r') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2087 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2088 *p++ = 'r';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2089 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2090
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2091 /* Map non-printable US ASCII to '\xhh' */
21067
c9c4c523dcbf Fix for bug #485951: repr diff between string and unicode.
Marc-André Lemburg <mal@egenix.com>
parents: 21028
diff changeset
2092 else if (ch < ' ' || ch >= 0x7F) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2093 *p++ = '\\';
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2094 *p++ = 'x';
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2095 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
2096 *p++ = hexdigit[ch & 0x000F];
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2097 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2098
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2099 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2100 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2101 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2102 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2103 if (quotes)
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
2104 *p++ = PyString_AS_STRING(repr)[1];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2105
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2106 *p = '\0';
23295
6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people
Tim Peters <tim.peters@gmail.com>
parents: 23294
diff changeset
2107 _PyString_Resize(&repr, p - PyString_AS_STRING(repr));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2108 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2109 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2110
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2111 PyObject *PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2112 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2113 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2114 return unicodeescape_string(s, size, 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2115 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2116
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2117 PyObject *PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2118 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2119 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2120 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2121 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2122 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2123 return PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2124 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2125 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2126
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2127 /* --- Raw Unicode Escape Codec ------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2128
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2129 PyObject *PyUnicode_DecodeRawUnicodeEscape(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2130 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2131 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2132 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2133 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2134 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2135 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2136 int outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2137 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2138 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2139 const char *end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2140 const char *bs;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2141 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2142 PyObject *exc = NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2143
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2144 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2145 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2146 length after conversion to the true value. (But decoding error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2147 handler might have to resize the string) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2148 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2149 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2150 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2151 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2152 return (PyObject *)v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2153 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2154 end = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2155 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2156 unsigned char c;
22423
b523a470a0ce Do not insert characters for unicode-escape decoders if the error mode
Martin v. Löwis <martin@v.loewis.de>
parents: 22087
diff changeset
2157 Py_UCS4 x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2158 int i;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2159 int count;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2160
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2161 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2162 if (*s != '\\') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2163 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2164 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2165 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2166 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2167
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2168 /* \u-escapes are only interpreted iff the number of leading
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2169 backslashes if odd */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2170 bs = s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2171 for (;s < end;) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2172 if (*s != '\\')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2173 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2174 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2175 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2176 if (((s - bs) & 1) == 0 ||
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2177 s >= end ||
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2178 (*s != 'u' && *s != 'U')) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2179 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2180 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2181 p--;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2182 count = *s=='u' ? 4 : 8;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2183 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2184
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2185 /* \uXXXX with 4 hex digits, \Uxxxxxxxx with 8 */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2186 outpos = p-PyUnicode_AS_UNICODE(v);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2187 for (x = 0, i = 0; i < count; ++i, ++s) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2188 c = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2189 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2190 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2191 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2192 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2193 "rawunicodeescape", "truncated \\uXXXX",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2194 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2195 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2196 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2197 goto nextByte;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2198 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2199 x = (x<<4) & ~0xF;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2200 if (c >= '0' && c <= '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2201 x += c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2202 else if (c >= 'a' && c <= 'f')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2203 x += 10 + c - 'a';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2204 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2205 x += 10 + c - 'A';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2206 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2207 #ifndef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2208 if (x > 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2209 if (unicode_decode_call_errorhandler(
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2210 errors, &errorHandler,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2211 "rawunicodeescape", "\\Uxxxxxxxx out of range",
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2212 starts, size, &startinpos, &endinpos, &exc, &s,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2213 (PyObject **)&v, &outpos, &p))
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2214 goto onError;
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2215 }
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2216 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2217 *p++ = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2218 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2219 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2220 }
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
2221 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))) < 0)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2222 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2223 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2224 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2225 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2226
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2227 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2228 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2229 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2230 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2231 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2232 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2233
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2234 PyObject *PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2235 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2236 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2237 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2238 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2239 char *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2240
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2241 static const char *hexdigit = "0123456789abcdef";
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2242
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2243 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2244 repr = PyString_FromStringAndSize(NULL, 10 * size);
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2245 #else
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2246 repr = PyString_FromStringAndSize(NULL, 6 * size);
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2247 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2248 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2249 return NULL;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2250 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2251 return repr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2252
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2253 p = q = PyString_AS_STRING(repr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2254 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2255 Py_UNICODE ch = *s++;
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2256 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2257 /* Map 32-bit characters to '\Uxxxxxxxx' */
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2258 if (ch >= 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2259 *p++ = '\\';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2260 *p++ = 'U';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2261 *p++ = hexdigit[(ch >> 28) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2262 *p++ = hexdigit[(ch >> 24) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2263 *p++ = hexdigit[(ch >> 20) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2264 *p++ = hexdigit[(ch >> 16) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2265 *p++ = hexdigit[(ch >> 12) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2266 *p++ = hexdigit[(ch >> 8) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2267 *p++ = hexdigit[(ch >> 4) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2268 *p++ = hexdigit[ch & 15];
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2269 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2270 else
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
2271 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2272 /* Map 16-bit characters to '\uxxxx' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2273 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2274 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2275 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2276 *p++ = hexdigit[(ch >> 12) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2277 *p++ = hexdigit[(ch >> 8) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2278 *p++ = hexdigit[(ch >> 4) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2279 *p++ = hexdigit[ch & 15];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2280 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2281 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2282 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2283 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2284 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2285 *p = '\0';
23295
6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people
Tim Peters <tim.peters@gmail.com>
parents: 23294
diff changeset
2286 _PyString_Resize(&repr, p - q);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2287 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2288 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2289
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2290 PyObject *PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2291 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2292 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2293 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2294 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2295 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2296 return PyUnicode_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2297 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2298 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2299
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2300 /* --- Unicode Internal Codec ------------------------------------------- */
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2301
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2302 PyObject *_PyUnicode_DecodeUnicodeInternal(const char *s,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2303 int size,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2304 const char *errors)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2305 {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2306 const char *starts = s;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2307 int startinpos;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2308 int endinpos;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2309 int outpos;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2310 Py_UNICODE unimax;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2311 PyUnicodeObject *v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2312 Py_UNICODE *p;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2313 const char *end;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2314 const char *reason;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2315 PyObject *errorHandler = NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2316 PyObject *exc = NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2317
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2318 unimax = PyUnicode_GetMax();
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2319 v = _PyUnicode_New((size+Py_UNICODE_SIZE-1)/ Py_UNICODE_SIZE);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2320 if (v == NULL)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2321 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2322 if (PyUnicode_GetSize((PyObject *)v) == 0)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2323 return (PyObject *)v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2324 p = PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2325 end = s + size;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2326
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2327 while (s < end) {
38183
fea5d2fd9b8a Backport: Patch #1488312, Fix memory alignment problem on SPARC in unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 37267
diff changeset
2328 memcpy(p, s, sizeof(Py_UNICODE));
35485
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2329 /* We have to sanity check the raw data, otherwise doom looms for
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2330 some malformed UCS-4 data. */
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2331 if (
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2332 #ifdef Py_UNICODE_WIDE
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2333 *p > unimax || *p < 0 ||
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2334 #endif
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2335 end-s < Py_UNICODE_SIZE
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2336 )
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2337 {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2338 startinpos = s - starts;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2339 if (end-s < Py_UNICODE_SIZE) {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2340 endinpos = end-starts;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2341 reason = "truncated input";
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2342 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2343 else {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2344 endinpos = s - starts + Py_UNICODE_SIZE;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2345 reason = "illegal code point (> 0x10FFFF)";
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2346 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2347 outpos = p - PyUnicode_AS_UNICODE(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2348 if (unicode_decode_call_errorhandler(
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2349 errors, &errorHandler,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2350 "unicode_internal", reason,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2351 starts, size, &startinpos, &endinpos, &exc, &s,
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2352 (PyObject **)&v, &outpos, &p)) {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2353 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2354 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2355 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2356 else {
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2357 p++;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2358 s += Py_UNICODE_SIZE;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2359 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2360 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2361
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2362 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))) < 0)
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2363 goto onError;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2364 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2365 Py_XDECREF(exc);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2366 return (PyObject *)v;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2367
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2368 onError:
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2369 Py_XDECREF(v);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2370 Py_XDECREF(errorHandler);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2371 Py_XDECREF(exc);
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2372 return NULL;
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2373 }
6b8c55877755 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 34016
diff changeset
2374
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2375 /* --- Latin-1 Codec ------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2376
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2377 PyObject *PyUnicode_DecodeLatin1(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2378 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2379 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2380 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2381 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2382 Py_UNICODE *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2383
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2384 /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2385 if (size == 1) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2386 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2387 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2388 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2389
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2390 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2391 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2392 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2393 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2394 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2395 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2396 while (size-- > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2397 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2398 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2399
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2400 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2401 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2402 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2403 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2404
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2405 /* create or adjust a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2406 static void make_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2407 const char *encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2408 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2409 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2410 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2411 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2412 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2413 *exceptionObject = PyUnicodeEncodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2414 encoding, unicode, size, startpos, endpos, reason);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2415 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2416 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2417 if (PyUnicodeEncodeError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2418 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2419 if (PyUnicodeEncodeError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2420 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2421 if (PyUnicodeEncodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2422 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2423 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2424 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2425 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2426 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2427 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2428 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2429
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2430 /* raises a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2431 static void raise_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2432 const char *encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2433 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2434 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2435 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2436 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2437 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2438 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2439 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2440 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2441 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2442
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2443 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2444 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2445 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2446 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2447 static PyObject *unicode_encode_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2448 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2449 const char *encoding, const char *reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2450 const Py_UNICODE *unicode, int size, PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2451 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2452 int *newpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2453 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2454 static char *argparse = "O!i;encoding error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2455
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2456 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2457 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2458
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2459 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2460 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2461 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2462 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2463 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2464
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2465 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2466 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2467 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2468 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2469
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2470 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2471 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2472 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2473 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2474 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2475 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2476 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2477 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2478 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2479 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2480 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2481 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2482 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2483 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2484 if (*newpos<0)
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
2485 *newpos = size+*newpos;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
2486 if (*newpos<0 || *newpos>size) {
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
2487 PyErr_Format(PyExc_IndexError, "position %d from error handler out of bounds", *newpos);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
2488 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
2489 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
2490 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2491 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2492 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2493 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2494 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2495
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2496 static PyObject *unicode_encode_ucs1(const Py_UNICODE *p,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2497 int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2498 const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2499 int limit)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2500 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2501 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2502 PyObject *res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2503 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2504 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2505 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2506 /* pointer to the beginning of the unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2507 /* const Py_UNICODE *badp = NULL; */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2508 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2509 char *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2510 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2511 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2512 int ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2513 char *encoding = (limit == 256) ? "latin-1" : "ascii";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2514 char *reason = (limit == 256) ? "ordinal not in range(256)" : "ordinal not in range(128)";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2515 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2516 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2517 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2518 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2519 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2520
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2521 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2522 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2523 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2524 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2525 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2526 if (size == 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2527 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2528 str = PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2529 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2530
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2531 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2532 Py_UNICODE c = *p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2533
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2534 /* can we encode this? */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2535 if (c<limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2536 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2537 *str++ = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2538 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2539 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2540 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2541 int unicodepos = p-startp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2542 int requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2543 PyObject *repunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2544 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2545 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2546 int respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2547 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2548 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2549 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2550 const Py_UNICODE *collend = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2551 /* find all unecodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2552 while ((collend < endp) && ((*collend)>=limit))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2553 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2554 /* cache callback name lookup (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2555 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2556 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2557 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2558 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2559 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2560 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2561 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2562 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2563 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2564 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2565 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2566 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2567 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2568 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2569 raise_encode_exception(&exc, encoding, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2570 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2571 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2572 while (collstart++<collend)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2573 *str++ = '?'; /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2574 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2575 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2576 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2577 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2578 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2579 /* determine replacement size (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2580 for (p = collstart, repsize = 0; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2581 if (*p<10)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2582 repsize += 2+1+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2583 else if (*p<100)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2584 repsize += 2+2+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2585 else if (*p<1000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2586 repsize += 2+3+1;
31248
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
2587 else if (*p<10000)
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
2588 repsize += 2+4+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2589 #ifndef Py_UNICODE_WIDE
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2590 else
31248
353a16033f6a Fix broken xmlcharrefreplace by rev 2.204.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31238
diff changeset
2591 repsize += 2+5+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2592 #else
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2593 else if (*p<100000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2594 repsize += 2+5+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2595 else if (*p<1000000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2596 repsize += 2+6+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2597 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2598 repsize += 2+7+1;
31238
4edf200a8118 SF #859573: Reduce compiler warnings on gcc 3.2 and above.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31199
diff changeset
2599 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2600 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2601 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2602 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2603 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2604 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2605 if (_PyString_Resize(&res, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2606 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2607 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2608 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2609 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2610 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2611 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2612 str += sprintf(str, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2613 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2614 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2615 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2616 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2617 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2618 encoding, reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2619 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2620 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2621 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2622 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2623 have+the replacement+the rest of the string, so
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2624 we won't have to check space for encodable characters) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2625 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2626 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2627 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2628 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2629 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2630 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2631 if (_PyString_Resize(&res, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2632 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2633 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2634 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2635 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2636 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2637 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2638 /* check if there is anything unencodable in the replacement
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2639 and copy it to the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2640 for (uni2 = PyUnicode_AS_UNICODE(repunicode);repsize-->0; ++uni2, ++str) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2641 c = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2642 if (c >= limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2643 raise_encode_exception(&exc, encoding, startp, size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2644 unicodepos, unicodepos+1, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2645 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2646 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2647 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2648 *str = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2649 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2650 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2651 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2652 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2653 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2654 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2655 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2656 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2657 if (respos<ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2658 /* If this falls res will be NULL */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2659 _PyString_Resize(&res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2660 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2661 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2662 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2663
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2664 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2665 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2666 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2667 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2668 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2669 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2670
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2671 PyObject *PyUnicode_EncodeLatin1(const Py_UNICODE *p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2672 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2673 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2674 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2675 return unicode_encode_ucs1(p, size, errors, 256);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2676 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2677
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2678 PyObject *PyUnicode_AsLatin1String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2679 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2680 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2681 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2682 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2683 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2684 return PyUnicode_EncodeLatin1(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2685 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2686 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2687 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2688
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2689 /* --- 7-bit ASCII Codec -------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2690
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2691 PyObject *PyUnicode_DecodeASCII(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2692 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2693 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2694 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2695 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2696 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2697 Py_UNICODE *p;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2698 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2699 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2700 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2701 const char *e;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2702 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2703 PyObject *exc = NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2704
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2705 /* ASCII is equivalent to the first 128 ordinals in Unicode. */
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2706 if (size == 1 && *(unsigned char*)s < 128) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2707 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2708 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2709 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2710
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2711 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2712 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2713 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2714 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2715 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2716 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2717 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2718 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2719 register unsigned char c = (unsigned char)*s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2720 if (c < 128) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2721 *p++ = c;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2722 ++s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2723 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2724 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2725 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2726 endinpos = startinpos + 1;
30439
30e495efe90a Double-fix of crash in Unicode freelist handling.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30431
diff changeset
2727 outpos = p - (Py_UNICODE *)PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2728 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2729 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2730 "ascii", "ordinal not in range(128)",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2731 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2732 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2733 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2734 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2735 }
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2736 if (p - PyUnicode_AS_UNICODE(v) < PyString_GET_SIZE(v))
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
2737 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))) < 0)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2738 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2739 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2740 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2741 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2742
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2743 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2744 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2745 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2746 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2747 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2748 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2749
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2750 PyObject *PyUnicode_EncodeASCII(const Py_UNICODE *p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2751 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2752 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2753 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2754 return unicode_encode_ucs1(p, size, errors, 128);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2755 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2756
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2757 PyObject *PyUnicode_AsASCIIString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2758 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2759 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2760 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2761 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2762 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2763 return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2764 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2765 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2766 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2767
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
2768 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
2769
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2770 /* --- MBCS codecs for Windows -------------------------------------------- */
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
2771
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2772 PyObject *PyUnicode_DecodeMBCS(const char *s,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2773 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2774 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2775 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2776 PyUnicodeObject *v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2777 Py_UNICODE *p;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2778
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2779 /* First get the size of the result */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2780 DWORD usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0);
12582
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2781 if (size > 0 && usize==0)
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2782 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2783
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2784 v = _PyUnicode_New(usize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2785 if (v == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2786 return NULL;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2787 if (usize == 0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2788 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2789 p = PyUnicode_AS_UNICODE(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2790 if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2791 Py_DECREF(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2792 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2793 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2794
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2795 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2796 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2797
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2798 PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2799 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2800 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2801 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2802 PyObject *repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2803 char *s;
12582
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2804 DWORD mbcssize;
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2805
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2806 /* If there are no characters, bail now! */
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2807 if (size==0)
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2808 return PyString_FromString("");
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2809
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2810 /* First get the size of the result */
12582
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2811 mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL);
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2812 if (mbcssize==0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2813 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2814
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2815 repr = PyString_FromStringAndSize(NULL, mbcssize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2816 if (repr == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2817 return NULL;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2818 if (mbcssize == 0)
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2819 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2820
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2821 /* Do the conversion */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2822 s = PyString_AS_STRING(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2823 if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2824 Py_DECREF(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2825 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2826 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2827 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2828 }
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
2829
29706
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2830 PyObject *PyUnicode_AsMBCSString(PyObject *unicode)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2831 {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2832 if (!PyUnicode_Check(unicode)) {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2833 PyErr_BadArgument();
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2834 return NULL;
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2835 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2836 return PyUnicode_EncodeMBCS(PyUnicode_AS_UNICODE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2837 PyUnicode_GET_SIZE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2838 NULL);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2839 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without
Mark Hammond <mhammond@skippinet.com.au>
parents: 29641
diff changeset
2840
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
2841 #endif /* MS_WINDOWS */
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2842
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2843 /* --- Character Mapping Codec -------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2844
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2845 PyObject *PyUnicode_DecodeCharmap(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2846 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2847 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2848 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2849 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2850 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2851 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2852 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2853 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2854 const char *e;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2855 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2856 Py_UNICODE *p;
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2857 int extrachars = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2858 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2859 PyObject *exc = NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2860
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2861 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2862 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2863 return PyUnicode_DecodeLatin1(s, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2864
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2865 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2866 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2867 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2868 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2869 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2870 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2871 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2872 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2873 unsigned char ch = *s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2874 PyObject *w, *x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2875
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2876 /* Get mapping (char ordinal -> integer, Unicode char or None) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2877 w = PyInt_FromLong((long)ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2878 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2879 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2880 x = PyObject_GetItem(mapping, w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2881 Py_DECREF(w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2882 if (x == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2883 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
15706
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2884 /* No mapping found means: mapping is undefined. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2885 PyErr_Clear();
15706
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2886 x = Py_None;
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2887 Py_INCREF(x);
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2888 } else
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2889 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2890 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2891
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2892 /* Apply mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2893 if (PyInt_Check(x)) {
13390
62754b9ec7df Fixed a couple of places where 'int' was used where 'long'
Marc-André Lemburg <mal@egenix.com>
parents: 13356
diff changeset
2894 long value = PyInt_AS_LONG(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2895 if (value < 0 || value > 65535) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2896 PyErr_SetString(PyExc_TypeError,
12909
be36f103a953 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12902
diff changeset
2897 "character mapping must be in range(65536)");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2898 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2899 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2900 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2901 *p++ = (Py_UNICODE)value;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2902 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2903 else if (x == Py_None) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2904 /* undefined mapping */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2905 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2906 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2907 endinpos = startinpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2908 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2909 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2910 "charmap", "character maps to <undefined>",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2911 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2912 (PyObject **)&v, &outpos, &p)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2913 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2914 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2915 }
35963
4d06162ad209 Backport checkin:
Walter Dörwald <walter@livinglogic.de>
parents: 35485
diff changeset
2916 Py_DECREF(x);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2917 continue;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2918 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2919 else if (PyUnicode_Check(x)) {
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2920 int targetsize = PyUnicode_GET_SIZE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2921
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2922 if (targetsize == 1)
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2923 /* 1-1 mapping */
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2924 *p++ = *PyUnicode_AS_UNICODE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2925
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2926 else if (targetsize > 1) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2927 /* 1-n mapping */
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2928 if (targetsize > extrachars) {
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2929 /* resize first */
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2930 int oldpos = (int)(p - PyUnicode_AS_UNICODE(v));
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2931 int needed = (targetsize - extrachars) + \
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2932 (targetsize << 2);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2933 extrachars += needed;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2934 if (_PyUnicode_Resize(&v,
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
2935 PyUnicode_GET_SIZE(v) + needed) < 0) {
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2936 Py_DECREF(x);
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2937 goto onError;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2938 }
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2939 p = PyUnicode_AS_UNICODE(v) + oldpos;
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2940 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2941 Py_UNICODE_COPY(p,
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2942 PyUnicode_AS_UNICODE(x),
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2943 targetsize);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2944 p += targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2945 extrachars -= targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2946 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2947 /* 1-0 mapping: skip the character */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2948 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2949 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2950 /* wrong return value */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2951 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2952 "character mapping must return integer, None or unicode");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2953 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2954 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2955 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2956 Py_DECREF(x);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2957 ++s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2958 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2959 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
2960 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2961 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2962 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2963 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2964 return (PyObject *)v;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
2965
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2966 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2967 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2968 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2969 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2970 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2971 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2972
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2973 /* Lookup the character ch in the mapping. If the character
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2974 can't be found, Py_None is returned (or NULL, if another
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2975 error occured). */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2976 static PyObject *charmapencode_lookup(Py_UNICODE c, PyObject *mapping)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2977 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2978 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2979 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2980
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2981 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2982 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2983 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2984 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2985 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2986 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2987 /* No mapping found means: mapping is undefined. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2988 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2989 x = Py_None;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2990 Py_INCREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2991 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2992 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2993 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2994 }
27175
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping
Walter Dörwald <walter@livinglogic.de>
parents: 27170
diff changeset
2995 else if (x == Py_None)
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping
Walter Dörwald <walter@livinglogic.de>
parents: 27170
diff changeset
2996 return x;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2997 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2998 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2999 if (value < 0 || value > 255) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3000 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3001 "character mapping must be in range(256)");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3002 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3003 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3004 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3005 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3006 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3007 else if (PyString_Check(x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3008 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3009 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3010 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3011 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3012 "character mapping must return integer, None or str");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3013 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3014 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3015 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3016 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3017
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3018 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3019 various state variables. Reallocate the output string if not enough
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3020 space is available. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3021 was put in the output buffer, or Py_None, if the mapping was undefined
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3022 (in which case no character was written) or NULL, if a
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3023 reallocation error ocurred. The called must decref the result */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3024 static
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3025 PyObject *charmapencode_output(Py_UNICODE c, PyObject *mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3026 PyObject **outobj, int *outpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3027 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3028 PyObject *rep = charmapencode_lookup(c, mapping);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3029
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3030 if (rep==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3031 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3032 else if (rep==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3033 return rep;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3034 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3035 char *outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3036 int outsize = PyString_GET_SIZE(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3037 if (PyInt_Check(rep)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3038 int requiredsize = *outpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3039 if (outsize<requiredsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3040 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3041 if (requiredsize < 2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3042 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3043 if (_PyString_Resize(outobj, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3044 Py_DECREF(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3045 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3046 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3047 outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3048 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3049 outstart[(*outpos)++] = (char)PyInt_AS_LONG(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3050 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3051 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3052 const char *repchars = PyString_AS_STRING(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3053 int repsize = PyString_GET_SIZE(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3054 int requiredsize = *outpos+repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3055 if (outsize<requiredsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3056 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3057 if (requiredsize < 2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3058 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3059 if (_PyString_Resize(outobj, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3060 Py_DECREF(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3061 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3062 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3063 outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3064 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3065 memcpy(outstart + *outpos, repchars, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3066 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3067 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3068 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3069 return rep;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3070 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3071
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3072 /* handle an error in PyUnicode_EncodeCharmap
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3073 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3074 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3075 int charmap_encoding_error(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3076 const Py_UNICODE *p, int size, int *inpos, PyObject *mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3077 PyObject **exceptionObject,
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
3078 int *known_errorHandler, PyObject **errorHandler, const char *errors,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3079 PyObject **res, int *respos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3080 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3081 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3082 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3083 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3084 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3085 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3086 int collstartpos = *inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3087 int collendpos = *inpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3088 int collpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3089 char *encoding = "charmap";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3090 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3092 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3093 /* find all unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3094 while (collendpos < size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3095 x = charmapencode_lookup(p[collendpos], mapping);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3096 if (x==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3097 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3098 else if (x!=Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3099 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3100 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3101 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3102 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3103 ++collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3104 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3105 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3106 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3107 if (*known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3108 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3109 *known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3110 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3111 *known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3112 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3113 *known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3114 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3115 *known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3116 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3117 *known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3118 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3119 switch (*known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3120 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3121 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3122 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3123 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3124 for (collpos = collstartpos; collpos<collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3125 x = charmapencode_output('?', mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3126 if (x==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3127 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3128 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3129 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3130 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3131 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3132 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3133 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3134 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3135 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3136 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3137 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3138 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3139 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3140 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3141 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3142 for (collpos = collstartpos; collpos < collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3143 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3144 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3145 sprintf(buffer, "&#%d;", (int)p[collpos]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3146 for (cp = buffer; *cp; ++cp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3147 x = charmapencode_output(*cp, mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3148 if (x==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3149 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3150 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3151 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3152 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3153 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3154 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3155 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3156 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3157 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3158 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3159 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3160 default:
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
3161 repunicode = unicode_encode_call_errorhandler(errors, errorHandler,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3162 encoding, reason, p, size, exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3163 collstartpos, collendpos, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3164 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3165 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3166 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3167 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3168 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3169 x = charmapencode_output(*uni2, mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3170 if (x==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3171 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3172 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3173 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3174 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3175 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3176 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3177 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3178 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3179 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3180 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3181 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3182 *inpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3183 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3184 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3185 return 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3186 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3187
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3188 PyObject *PyUnicode_EncodeCharmap(const Py_UNICODE *p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3189 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3190 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3191 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3192 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3193 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3194 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3195 /* current input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3196 int inpos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3197 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3198 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3199 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3200 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3201 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3202 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3203 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3204 int known_errorHandler = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3205
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3206 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3207 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3208 return PyUnicode_EncodeLatin1(p, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3209
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3210 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3211 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3212 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3213 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3214 goto onError;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
3215 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3216 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3217
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3218 while (inpos<size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3219 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3220 PyObject *x = charmapencode_output(p[inpos], mapping, &res, &respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3221 if (x==NULL) /* error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3222 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3223 if (x==Py_None) { /* unencodable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3224 if (charmap_encoding_error(p, size, &inpos, mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3225 &exc,
30149
59bad7efff02 Fix refcount leak in PyUnicode_EncodeCharmap(). The bug surfaces
Walter Dörwald <walter@livinglogic.de>
parents: 29706
diff changeset
3226 &known_errorHandler, &errorHandler, errors,
30162
443533d08783 Fix another refcounting leak in PyUnicode_EncodeCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30160
diff changeset
3227 &res, &respos)) {
443533d08783 Fix another refcounting leak in PyUnicode_EncodeCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30160
diff changeset
3228 Py_DECREF(x);
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3229 goto onError;
30162
443533d08783 Fix another refcounting leak in PyUnicode_EncodeCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30160
diff changeset
3230 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3231 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3232 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3233 /* done with this character => adjust input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3234 ++inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3235 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3236 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3237
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3238 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3239 if (respos<PyString_GET_SIZE(res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3240 if (_PyString_Resize(&res, respos))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3241 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3242 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3243 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3244 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3245 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3246
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3247 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3248 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3249 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3250 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3251 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3252 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3253
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3254 PyObject *PyUnicode_AsCharmapString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3255 PyObject *mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3256 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3257 if (!PyUnicode_Check(unicode) || mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3258 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3259 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3260 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3261 return PyUnicode_EncodeCharmap(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3262 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3263 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3264 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3265 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3266
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3267 /* create or adjust a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3268 static void make_translate_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3269 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3270 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3271 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3272 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3273 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3274 *exceptionObject = PyUnicodeTranslateError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3275 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3276 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3277 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3278 if (PyUnicodeTranslateError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3279 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3280 if (PyUnicodeTranslateError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3281 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3282 if (PyUnicodeTranslateError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3283 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3284 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3285 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3286 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3287 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3288 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3289 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3290
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3291 /* raises a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3292 static void raise_translate_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3293 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3294 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3295 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3296 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3297 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3298 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3299 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3300 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3301 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3302
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3303 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3304 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3305 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3306 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3307 static PyObject *unicode_translate_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3308 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3309 const char *reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3310 const Py_UNICODE *unicode, int size, PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3311 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3312 int *newpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3313 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3314 static char *argparse = "O!i;translating error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3315
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3316 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3317 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3318
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3319 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3320 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3321 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3322 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3323 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3324
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3325 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3326 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3327 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3328 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3329
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3330 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3331 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3332 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3333 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3334 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3335 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3336 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3337 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3338 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3339 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3340 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3341 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3342 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3343 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3344 if (*newpos<0)
27555
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3345 *newpos = size+*newpos;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3346 if (*newpos<0 || *newpos>size) {
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3347 PyErr_Format(PyExc_IndexError, "position %d from error handler out of bounds", *newpos);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3348 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3349 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293
Walter Dörwald <walter@livinglogic.de>
parents: 27521
diff changeset
3350 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3351 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3352 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3353 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3354 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3355
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3356 /* Lookup the character ch in the mapping and put the result in result,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3357 which must be decrefed by the caller.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3358 Return 0 on success, -1 on error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3359 static
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3360 int charmaptranslate_lookup(Py_UNICODE c, PyObject *mapping, PyObject **result)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3361 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3362 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3363 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3364
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3365 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3366 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3367 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3368 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3369 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3370 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3371 /* No mapping found means: use 1:1 mapping. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3372 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3373 *result = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3374 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3375 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3376 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3377 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3378 else if (x == Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3379 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3380 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3381 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3382 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3383 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3384 long max = PyUnicode_GetMax();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3385 if (value < 0 || value > max) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3386 PyErr_Format(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3387 "character mapping must be in range(0x%lx)", max+1);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3388 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3389 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3390 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3391 *result = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3392 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3393 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3394 else if (PyUnicode_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3395 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3396 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3397 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3398 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3399 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3400 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3401 "character mapping must return integer, None or unicode");
30164
7cb4ad5932e6 Fix refcounting leak in charmaptranslate_lookup()
Walter Dörwald <walter@livinglogic.de>
parents: 30162
diff changeset
3402 Py_DECREF(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3403 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3404 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3405 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3406 /* ensure that *outobj is at least requiredsize characters long,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3407 if not reallocate and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3408 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3409 static
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3410 int charmaptranslate_makespace(PyObject **outobj, Py_UNICODE **outp,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3411 int requiredsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3412 {
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3413 int oldsize = PyUnicode_GET_SIZE(*outobj);
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3414 if (requiredsize > oldsize) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3415 /* remember old output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3416 int outpos = *outp-PyUnicode_AS_UNICODE(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3417 /* exponentially overallocate to minimize reallocations */
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3418 if (requiredsize < 2 * oldsize)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3419 requiredsize = 2 * oldsize;
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
3420 if (_PyUnicode_Resize(outobj, requiredsize) < 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3421 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3422 *outp = PyUnicode_AS_UNICODE(*outobj) + outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3423 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3424 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3425 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3426 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3427 various state variables. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3428 was put in the output buffer in *result, or Py_None, if the mapping was
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3429 undefined (in which case no character was written).
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3430 The called must decref result.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3431 Return 0 on success, -1 on error. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3432 static
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3433 int charmaptranslate_output(const Py_UNICODE *startinp, const Py_UNICODE *curinp,
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3434 int insize, PyObject *mapping, PyObject **outobj, Py_UNICODE **outp,
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3435 PyObject **res)
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3436 {
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3437 if (charmaptranslate_lookup(*curinp, mapping, res))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3438 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3439 if (*res==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3440 /* not found => default to 1:1 mapping */
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3441 *(*outp)++ = *curinp;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3442 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3443 else if (*res==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3444 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3445 else if (PyInt_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3446 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3447 *(*outp)++ = (Py_UNICODE)PyInt_AS_LONG(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3448 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3449 else if (PyUnicode_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3450 int repsize = PyUnicode_GET_SIZE(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3451 if (repsize==1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3452 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3453 *(*outp)++ = *PyUnicode_AS_UNICODE(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3454 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3455 else if (repsize!=0) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3456 /* more than one character */
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3457 int requiredsize = (*outp-PyUnicode_AS_UNICODE(*outobj)) +
31455
37d501b9e2d5 Fix reallocation bug in unicode.translate(): The code was comparing
Walter Dörwald <walter@livinglogic.de>
parents: 31325
diff changeset
3458 (insize - (curinp-startinp)) +
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3459 repsize - 1;
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3460 if (charmaptranslate_makespace(outobj, outp, requiredsize))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3461 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3462 memcpy(*outp, PyUnicode_AS_UNICODE(*res), sizeof(Py_UNICODE)*repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3463 *outp += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3464 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3465 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3466 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3467 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3468 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3469 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3470
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3471 PyObject *PyUnicode_TranslateCharmap(const Py_UNICODE *p,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3472 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3473 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3474 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3475 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3476 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3477 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3478 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3479 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3480 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3481 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3482 Py_UNICODE *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3483 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3484 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3485 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3486 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3487 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3488 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3489 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3490 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3491 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3492
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3493 if (mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3494 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3495 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3496 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3497
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3498 /* allocate enough for a simple 1:1 translation without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3499 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3500 res = PyUnicode_FromUnicode(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3501 if (res == NULL)
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3502 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3503 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3504 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3505 str = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3506
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3507 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3508 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3509 PyObject *x = NULL;
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3510 if (charmaptranslate_output(startp, p, size, mapping, &res, &str, &x)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3511 Py_XDECREF(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3512 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3513 }
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
3514 Py_XDECREF(x);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3515 if (x!=Py_None) /* it worked => adjust input pointer */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3516 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3517 else { /* untranslatable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3518 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3519 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3520 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3521 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3522 /* startpos for collecting untranslatable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3523 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3524 const Py_UNICODE *collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3525 const Py_UNICODE *coll;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3526
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3527 /* find all untranslatable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3528 while (collend < endp) {
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3529 if (charmaptranslate_lookup(*collend, mapping, &x))
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3530 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3531 Py_XDECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3532 if (x!=Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3533 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3534 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3535 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3536 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3537 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3538 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3539 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3540 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3541 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3542 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3543 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3544 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3545 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3546 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3547 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3548 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3549 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3550 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3551 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3552 raise_translate_exception(&exc, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3553 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3554 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3555 /* No need to check for space, this is a 1:1 replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3556 for (coll = collstart; coll<collend; ++coll)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3557 *str++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3558 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3559 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3560 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3561 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3562 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3563 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3564 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3565 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3566 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3567 sprintf(buffer, "&#%d;", (int)*p);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3568 if (charmaptranslate_makespace(&res, &str,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3569 (str-PyUnicode_AS_UNICODE(res))+strlen(buffer)+(endp-collend)))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3570 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3571 for (cp = buffer; *cp; ++cp)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3572 *str++ = *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3573 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3574 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3575 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3576 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3577 repunicode = unicode_translate_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3578 reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3579 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3580 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3581 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3582 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3583 repsize = PyUnicode_GET_SIZE(repunicode);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3584 if (charmaptranslate_makespace(&res, &str,
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3585 (str-PyUnicode_AS_UNICODE(res))+repsize+(endp-collend))) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3586 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3587 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3588 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3589 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3590 *str++ = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3591 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3592 Py_DECREF(repunicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3593 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3594 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3595 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3596 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3597 respos = str-PyUnicode_AS_UNICODE(res);
30765
fe3f5592305d Fix a bug in the memory reallocation code of PyUnicode_TranslateCharmap().
Walter Dörwald <walter@livinglogic.de>
parents: 30707
diff changeset
3598 if (respos<PyUnicode_GET_SIZE(res)) {
30431
cd0c3f916b34 Change checks of PyUnicode_Resize() return value for clarity.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 30216
diff changeset
3599 if (_PyUnicode_Resize(&res, respos) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3600 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3601 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3602 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3603 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3604 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3605
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3606 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3607 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3608 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3609 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3610 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3611 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3613 PyObject *PyUnicode_Translate(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3614 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3615 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3616 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3617 PyObject *result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3618
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3619 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3620 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3621 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3622 result = PyUnicode_TranslateCharmap(PyUnicode_AS_UNICODE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3623 PyUnicode_GET_SIZE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3624 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3625 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3626 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3627 return result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3628
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3629 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3630 Py_XDECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3631 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3632 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3633
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3634 /* --- Decimal Encoder ---------------------------------------------------- */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3635
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3636 int PyUnicode_EncodeDecimal(Py_UNICODE *s,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3637 int length,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3638 char *output,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3639 const char *errors)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3640 {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3641 Py_UNICODE *p, *end;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3642 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3643 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3644 const char *encoding = "decimal";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3645 const char *reason = "invalid decimal Unicode string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3646 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3647 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3648 int known_errorHandler = -1;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3649
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3650 if (output == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3651 PyErr_BadArgument();
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3652 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3653 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3654
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3655 p = s;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3656 end = s + length;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3657 while (p < end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3658 register Py_UNICODE ch = *p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3659 int decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3660 PyObject *repunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3661 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3662 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3663 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3664 Py_UNICODE *collstart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3665 Py_UNICODE *collend;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3666
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3667 if (Py_UNICODE_ISSPACE(ch)) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3668 *output++ = ' ';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3669 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3670 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3671 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3672 decimal = Py_UNICODE_TODECIMAL(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3673 if (decimal >= 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3674 *output++ = '0' + decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3675 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3676 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3677 }
12275
cbfa95a6a03e Conrad Huang points out that "if (0 < ch < 256)", while legal C,
Guido van Rossum <guido@python.org>
parents: 12249
diff changeset
3678 if (0 < ch && ch < 256) {
12577
85f00f8cd674 Fix warning detected by VC++ on assignment of Py_UNICODE to char.
Guido van Rossum <guido@python.org>
parents: 12576
diff changeset
3679 *output++ = (char)ch;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3680 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3681 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3682 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3683 /* All other characters are considered unencodable */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3684 collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3685 collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3686 while (collend < end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3687 if ((0 < *collend && *collend < 256) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3688 !Py_UNICODE_ISSPACE(*collend) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3689 Py_UNICODE_TODECIMAL(*collend))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3690 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3691 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3692 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3693 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3694 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3695 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3696 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3697 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3698 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3699 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3700 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3701 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3702 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3703 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3704 known_errorHandler = 0;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3705 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3706 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3707 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3708 raise_encode_exception(&exc, encoding, s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3709 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3710 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3711 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3712 *output++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3713 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3714 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3715 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3716 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3717 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3718 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3719 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3720 output += sprintf(output, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3721 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3722 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3723 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3724 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3725 encoding, reason, s, length, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3726 collstart-s, collend-s, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3727 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3728 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3729 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3730 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3731 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3732 Py_UNICODE ch = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3733 if (Py_UNICODE_ISSPACE(ch))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3734 *output++ = ' ';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3735 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3736 decimal = Py_UNICODE_TODECIMAL(ch);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3737 if (decimal >= 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3738 *output++ = '0' + decimal;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3739 else if (0 < ch && ch < 256)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3740 *output++ = (char)ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3741 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3742 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3743 raise_encode_exception(&exc, encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3744 s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3745 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3746 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3747 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3748 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3749 p = s + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3750 Py_DECREF(repunicode);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3751 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3752 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3753 /* 0-terminate the output string */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3754 *output++ = '\0';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3755 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3756 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3757 return 0;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3758
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3759 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3760 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3761 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3762 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3763 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3764
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3765 /* --- Helpers ------------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3766
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3767 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3768 int count(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3769 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3770 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3771 PyUnicodeObject *substring)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3772 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3773 int count = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3774
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3775 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3776 start += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3777 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3778 start = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3779 if (end > self->length)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3780 end = self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3781 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3782 end += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3783 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3784 end = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3785
12957
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3786 if (substring->length == 0)
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3787 return (end - start + 1);
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3788
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3789 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3790
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3791 while (start <= end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3792 if (Py_UNICODE_MATCH(self, start, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3793 count++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3794 start += substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3795 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3796 start++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3797
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3798 return count;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3799 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3800
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3801 int PyUnicode_Count(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3802 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3803 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3804 int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3805 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3806 int result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3807
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3808 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3809 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3810 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3811 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3812 if (substr == NULL) {
12957
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3813 Py_DECREF(str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3814 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3815 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3816
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3817 result = count((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3818 start, end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3819 (PyUnicodeObject *)substr);
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3820
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3821 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3822 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3823 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3824 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3825
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3826 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3827 int findstring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3828 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3829 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3830 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3831 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3832 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3833 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3834 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3835 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3836 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3837
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3838 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3839 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3840 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3841 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3842 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3843 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3844
24918
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith().
Guido van Rossum <guido@python.org>
parents: 24831
diff changeset
3845 if (substring->length == 0)
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith().
Guido van Rossum <guido@python.org>
parents: 24831
diff changeset
3846 return (direction > 0) ? start : end;
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith().
Guido van Rossum <guido@python.org>
parents: 24831
diff changeset
3847
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3848 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3849
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3850 if (direction < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3851 for (; end >= start; end--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3852 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3853 return end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3854 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3855 for (; start <= end; start++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3856 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3857 return start;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3858 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3859
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3860 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3861 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3862
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3863 int PyUnicode_Find(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3864 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3865 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3866 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3867 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3868 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3869 int result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3870
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3871 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3872 if (str == NULL)
23561
c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents: 23542
diff changeset
3873 return -2;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3874 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3875 if (substr == NULL) {
23562
eaca627f2a54 Fix a possible segfault. Found be Neal Norvitz.
Marc-André Lemburg <mal@egenix.com>
parents: 23561
diff changeset
3876 Py_DECREF(str);
23561
c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents: 23542
diff changeset
3877 return -2;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3878 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3879
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3880 result = findstring((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3881 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3882 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3883 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3884 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3885 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3886 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3887
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3888 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3889 int tailmatch(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3890 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3891 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3892 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3893 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3894 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3895 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3896 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3897 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3898 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3899
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3900 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3901 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3902
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3903 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3904 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3905 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3906 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3907 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3908 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3909
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3910 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3911 if (end < start)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3912 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3913
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3914 if (direction > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3915 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3916 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3917 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3918 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3919 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3920 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3921
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3922 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3923 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3924
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3925 int PyUnicode_Tailmatch(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3926 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3927 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3928 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3929 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3930 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3931 int result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3932
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3933 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3934 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3935 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3936 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3937 if (substr == NULL) {
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
3938 Py_DECREF(str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3939 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3940 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3941
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3942 result = tailmatch((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3943 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3944 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3945 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3946 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3947 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3948 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3949
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3950 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3951 const Py_UNICODE *findchar(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3952 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3953 Py_UNICODE ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3954 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3955 /* like wcschr, but doesn't stop at NULL characters */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3956
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3957 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3958 if (*s == ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3959 return s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3960 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3961 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3962
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3963 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3964 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3965
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3966 /* Apply fixfct filter to the Unicode object self and return a
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3967 reference to the modified object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3968
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3969 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3970 PyObject *fixup(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3971 int (*fixfct)(PyUnicodeObject *s))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3972 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3973
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3974 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3975
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3976 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3977 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3978 return NULL;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3979
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3980 Py_UNICODE_COPY(u->str, self->str, self->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3981
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
3982 if (!fixfct(u) && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3983 /* fixfct should return TRUE if it modified the buffer. If
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3984 FALSE, return a reference to the original buffer instead
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3985 (to save space, not time) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3986 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3987 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3988 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3989 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3990 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3991 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3992
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3993 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3994 int fixupper(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3995 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3996 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3997 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3998 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
3999
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4000 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4001 register Py_UNICODE ch;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4002
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4003 ch = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4004 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4005 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4006 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4007 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4008 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4009 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4010
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4011 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4012 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4013
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4014 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4015 int fixlower(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4016 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4017 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4018 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4019 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4020
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4021 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4022 register Py_UNICODE ch;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4023
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4024 ch = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4025 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4026 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4027 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4028 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4029 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4030 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4031
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4032 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4033 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4034
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4035 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4036 int fixswapcase(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4037 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4038 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4039 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4040 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4041
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4042 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4043 if (Py_UNICODE_ISUPPER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4044 *s = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4045 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4046 } else if (Py_UNICODE_ISLOWER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4047 *s = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4048 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4049 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4050 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4051 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4052
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4053 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4054 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4055
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4056 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4057 int fixcapitalize(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4058 {
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4059 int len = self->length;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4060 Py_UNICODE *s = self->str;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4061 int status = 0;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4062
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4063 if (len == 0)
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4064 return 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4065 if (Py_UNICODE_ISLOWER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4066 *s = Py_UNICODE_TOUPPER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4067 status = 1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4068 }
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4069 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4070 while (--len > 0) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4071 if (Py_UNICODE_ISUPPER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4072 *s = Py_UNICODE_TOLOWER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4073 status = 1;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4074 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4075 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4076 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
4077 return status;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4078 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4079
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4080 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4081 int fixtitle(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4082 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4083 register Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4084 register Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4085 int previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4086
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4087 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4088 if (PyUnicode_GET_SIZE(self) == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4089 Py_UNICODE ch = Py_UNICODE_TOTITLE(*p);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4090 if (*p != ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4091 *p = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4092 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4093 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4094 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4095 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4096 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4097
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4098 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4099 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4100 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4101 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4102
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4103 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4104 *p = Py_UNICODE_TOLOWER(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4105 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4106 *p = Py_UNICODE_TOTITLE(ch);
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4107
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4108 if (Py_UNICODE_ISLOWER(ch) ||
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4109 Py_UNICODE_ISUPPER(ch) ||
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4110 Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4111 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4112 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4113 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4114 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4115 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4116 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4117
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4118 PyObject *
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4119 PyUnicode_Join(PyObject *separator, PyObject *seq)
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4120 {
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4121 PyObject *internal_separator = NULL;
33499
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc.
Skip Montanaro <skip@pobox.com>
parents: 33495
diff changeset
4122 const Py_UNICODE blank = ' ';
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc.
Skip Montanaro <skip@pobox.com>
parents: 33495
diff changeset
4123 const Py_UNICODE *sep = &blank;
a4d3673bcd66 Initialize sep and seplen to suppress warning from gcc.
Skip Montanaro <skip@pobox.com>
parents: 33495
diff changeset
4124 size_t seplen = 1;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4125 PyUnicodeObject *res = NULL; /* the result */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4126 size_t res_alloc = 100; /* # allocated bytes for string in res */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4127 size_t res_used; /* # used bytes */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4128 Py_UNICODE *res_p; /* pointer to free byte in res's string area */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4129 PyObject *fseq; /* PySequence_Fast(seq) */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4130 int seqlen; /* len(fseq) -- number of items in sequence */
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4131 PyObject *item;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4132 int i;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4133
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4134 fseq = PySequence_Fast(seq, "");
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4135 if (fseq == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4136 if (PyErr_ExceptionMatches(PyExc_TypeError))
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4137 PyErr_Format(PyExc_TypeError,
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4138 "sequence expected, %.80s found",
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4139 seq->ob_type->tp_name);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4140 return NULL;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4141 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4142
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4143 /* Grrrr. A codec may be invoked to convert str objects to
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4144 * Unicode, and so it's possible to call back into Python code
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4145 * during PyUnicode_FromObject(), and so it's possible for a sick
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4146 * codec to change the size of fseq (if seq is a list). Therefore
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4147 * we have to keep refetching the size -- can't assume seqlen
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4148 * is invariant.
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4149 */
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4150 seqlen = PySequence_Fast_GET_SIZE(fseq);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4151 /* If empty sequence, return u"". */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4152 if (seqlen == 0) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4153 res = _PyUnicode_New(0); /* empty sequence; return u"" */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4154 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4155 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4156 /* If singleton sequence with an exact Unicode, return that. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4157 if (seqlen == 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4158 item = PySequence_Fast_GET_ITEM(fseq, 0);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4159 if (PyUnicode_CheckExact(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4160 Py_INCREF(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4161 res = (PyUnicodeObject *)item;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4162 goto Done;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4163 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4164 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4165
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4166 /* At least two items to join, or one that isn't exact Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4167 if (seqlen > 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4168 /* Set up sep and seplen -- they're needed. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4169 if (separator == NULL) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4170 sep = &blank;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4171 seplen = 1;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4172 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4173 else {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4174 internal_separator = PyUnicode_FromObject(separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4175 if (internal_separator == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4176 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4177 sep = PyUnicode_AS_UNICODE(internal_separator);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4178 seplen = PyUnicode_GET_SIZE(internal_separator);
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4179 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4180 seqlen = PySequence_Fast_GET_SIZE(fseq);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4181 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4182 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4183
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4184 /* Get space. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4185 res = _PyUnicode_New((int)res_alloc);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4186 if (res == NULL)
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4187 goto onError;
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4188 res_p = PyUnicode_AS_UNICODE(res);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4189 res_used = 0;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4190
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4191 for (i = 0; i < seqlen; ++i) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4192 size_t itemlen;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4193 size_t new_res_used;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4194
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4195 item = PySequence_Fast_GET_ITEM(fseq, i);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4196 /* Convert item to Unicode. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4197 if (! PyUnicode_Check(item) && ! PyString_Check(item)) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4198 PyErr_Format(PyExc_TypeError,
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4199 "sequence item %i: expected string or Unicode,"
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4200 " %.80s found",
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4201 i, item->ob_type->tp_name);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
4202 goto onError;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
4203 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4204 item = PyUnicode_FromObject(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4205 if (item == NULL)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4206 goto onError;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4207 /* We own a reference to item from here on. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4208
33311
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4209 /* In case PyUnicode_FromObject() mutated seq. */
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4210 seqlen = PySequence_Fast_GET_SIZE(fseq);
59bd121eeeb7 PyUnicode_Join(): Bozo Alert. While this is chugging along, it may
Tim Peters <tim.peters@gmail.com>
parents: 33310
diff changeset
4211
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4212 /* Make sure we have enough space for the separator and the item. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4213 itemlen = PyUnicode_GET_SIZE(item);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4214 new_res_used = res_used + itemlen;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4215 if (new_res_used < res_used || new_res_used > INT_MAX)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4216 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4217 if (i < seqlen - 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4218 new_res_used += seplen;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4219 if (new_res_used < res_used || new_res_used > INT_MAX)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4220 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4221 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4222 if (new_res_used > res_alloc) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4223 /* double allocated size until it's big enough */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4224 do {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4225 size_t oldsize = res_alloc;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4226 res_alloc += res_alloc;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4227 if (res_alloc < oldsize || res_alloc > INT_MAX)
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4228 goto Overflow;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4229 } while (new_res_used > res_alloc);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4230 if (_PyUnicode_Resize(&res, (int)res_alloc) < 0) {
20007
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
4231 Py_DECREF(item);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
4232 goto onError;
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
4233 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4234 res_p = PyUnicode_AS_UNICODE(res) + res_used;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4235 }
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4236
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4237 /* Copy item, and maybe the separator. */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4238 Py_UNICODE_COPY(res_p, PyUnicode_AS_UNICODE(item), (int)itemlen);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4239 res_p += itemlen;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4240 if (i < seqlen - 1) {
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4241 Py_UNICODE_COPY(res_p, sep, (int)seplen);
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4242 res_p += seplen;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4243 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4244 Py_DECREF(item);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4245 res_used = new_res_used;
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4246 }
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4247
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4248 /* Shrink res to match the used area; this probably can't fail,
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4249 * but it's cheap to check.
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4250 */
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4251 if (_PyUnicode_Resize(&res, (int)res_used) < 0)
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4252 goto onError;
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4253
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4254 Done:
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4255 Py_XDECREF(internal_separator);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4256 Py_DECREF(fseq);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4257 return (PyObject *)res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4258
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4259 Overflow:
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4260 PyErr_SetString(PyExc_OverflowError,
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4261 "join() is too long for a Python string");
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4262 Py_DECREF(item);
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4263 /* fall through */
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4264
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4265 onError:
33300
77de38534d26 PyUnicode_Join(): Two primary aims:
Tim Peters <tim.peters@gmail.com>
parents: 32951
diff changeset
4266 Py_XDECREF(internal_separator);
33310
8579859f198c PyUnicode_Join(): Rewrote to use PySequence_Fast(). This doesn't do
Tim Peters <tim.peters@gmail.com>
parents: 33304
diff changeset
4267 Py_DECREF(fseq);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
4268 Py_XDECREF(res);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4269 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4270 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4271
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4272 static
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4273 PyUnicodeObject *pad(PyUnicodeObject *self,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4274 int left,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4275 int right,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4276 Py_UNICODE fill)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4277 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4278 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4279
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4280 if (left < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4281 left = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4282 if (right < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4283 right = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4284
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4285 if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4286 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4287 return self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4288 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4289
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4290 u = _PyUnicode_New(left + self->length + right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4291 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4292 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4293 Py_UNICODE_FILL(u->str, fill, left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4294 Py_UNICODE_COPY(u->str + left, self->str, self->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4295 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4296 Py_UNICODE_FILL(u->str + left + self->length, fill, right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4297 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4298
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4299 return u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4300 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4301
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4302 #define SPLIT_APPEND(data, left, right) \
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4303 str = PyUnicode_FromUnicode((data) + (left), (right) - (left)); \
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4304 if (!str) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4305 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4306 if (PyList_Append(list, str)) { \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4307 Py_DECREF(str); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4308 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4309 } \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4310 else \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4311 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4312
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4313 #define SPLIT_INSERT(data, left, right) \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4314 str = PyUnicode_FromUnicode((data) + (left), (right) - (left)); \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4315 if (!str) \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4316 goto onError; \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4317 if (PyList_Insert(list, 0, str)) { \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4318 Py_DECREF(str); \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4319 goto onError; \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4320 } \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4321 else \
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4322 Py_DECREF(str);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4323
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4324 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4325 PyObject *split_whitespace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4326 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4327 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4328 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4329 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4330 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4331 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4332 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4333
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4334 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4335 /* find a token */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4336 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4337 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4338 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4339 while (i < len && !Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4340 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4341 if (j < i) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4342 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4343 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4344 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4345 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4346 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4347 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4348 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4349 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4350 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4351 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4352 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4353 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4354
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4355 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4356 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4357 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4358 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4359
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4360 PyObject *PyUnicode_Splitlines(PyObject *string,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4361 int keepends)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4362 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4363 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4364 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4365 int len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4366 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4367 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4368 Py_UNICODE *data;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4369
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4370 string = PyUnicode_FromObject(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4371 if (string == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4372 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4373 data = PyUnicode_AS_UNICODE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4374 len = PyUnicode_GET_SIZE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4375
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4376 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4377 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4378 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4379
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4380 for (i = j = 0; i < len; ) {
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4381 int eol;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4382
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4383 /* Find a line and append it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4384 while (i < len && !Py_UNICODE_ISLINEBREAK(data[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4385 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4386
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4387 /* Skip the line break reading CRLF as one line break */
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4388 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4389 if (i < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4390 if (data[i] == '\r' && i + 1 < len &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4391 data[i+1] == '\n')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4392 i += 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4393 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4394 i++;
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4395 if (keepends)
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4396 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4397 }
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4398 SPLIT_APPEND(data, j, eol);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4399 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4400 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4401 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4402 SPLIT_APPEND(data, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4403 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4404
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4405 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4406 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4407
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4408 onError:
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
4409 Py_XDECREF(list);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4410 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4411 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4412 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4413
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4414 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4415 PyObject *split_char(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4416 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4417 Py_UNICODE ch,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4418 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4419 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4420 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4421 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4422 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4423 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4424
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4425 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4426 if (self->str[i] == ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4427 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4428 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4429 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4430 i = j = i + 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4431 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4432 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4433 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4434 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4435 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4436 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4437 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4438
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4439 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4440 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4441 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4442 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4443
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4444 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4445 PyObject *split_substring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4446 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4447 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4448 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4449 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4450 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4451 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4452 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4453 int sublen = substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4454 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4455
15609
0906c8e2c961 Fix off-by-one error in split_substring(). Fixes SF bug #122162.
Guido van Rossum <guido@python.org>
parents: 15579
diff changeset
4456 for (i = j = 0; i <= len - sublen; ) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4457 if (Py_UNICODE_MATCH(self, i, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4458 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4459 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4460 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4461 i = j = i + sublen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4462 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4463 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4464 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4465 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4466 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4467 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4468 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4469
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4470 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4471 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4472 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4473 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4474
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4475 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4476 PyObject *rsplit_whitespace(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4477 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4478 int maxcount)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4479 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4480 register int i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4481 register int j;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4482 int len = self->length;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4483 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4484
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4485 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4486 /* find a token */
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4487 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4488 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4489 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4490 while (i >= 0 && !Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4491 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4492 if (j > i) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4493 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4494 break;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4495 SPLIT_INSERT(self->str, i + 1, j + 1);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4496 while (i >= 0 && Py_UNICODE_ISSPACE(self->str[i]))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4497 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4498 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4499 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4500 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4501 if (j >= 0) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4502 SPLIT_INSERT(self->str, 0, j + 1);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4503 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4504 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4505
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4506 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4507 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4508 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4509 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4510
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4511 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4512 PyObject *rsplit_char(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4513 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4514 Py_UNICODE ch,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4515 int maxcount)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4516 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4517 register int i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4518 register int j;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4519 int len = self->length;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4520 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4521
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4522 for (i = j = len - 1; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4523 if (self->str[i] == ch) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4524 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4525 break;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4526 SPLIT_INSERT(self->str, i + 1, j + 1);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4527 j = i = i - 1;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4528 } else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4529 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4530 }
31252
892116a886e0 Fix unicode.rsplit()'s bug that ignores separater on the end of string when
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31248
diff changeset
4531 if (j >= -1) {
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4532 SPLIT_INSERT(self->str, 0, j + 1);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4533 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4534 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4535
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4536 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4537 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4538 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4539 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4540
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4541 static
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4542 PyObject *rsplit_substring(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4543 PyObject *list,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4544 PyUnicodeObject *substring,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4545 int maxcount)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4546 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4547 register int i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4548 register int j;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4549 int len = self->length;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4550 int sublen = substring->length;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4551 PyObject *str;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4552
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4553 for (i = len - sublen, j = len; i >= 0; ) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4554 if (Py_UNICODE_MATCH(self, i, substring)) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4555 if (maxcount-- <= 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4556 break;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4557 SPLIT_INSERT(self->str, i + sublen, j);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4558 j = i;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4559 i -= sublen;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4560 } else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4561 i--;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4562 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4563 if (j >= 0) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4564 SPLIT_INSERT(self->str, 0, j);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4565 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4566 return list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4567
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4568 onError:
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4569 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4570 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4571 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4572
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4573 #undef SPLIT_APPEND
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4574 #undef SPLIT_INSERT
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4575
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4576 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4577 PyObject *split(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4578 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4579 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4580 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4581 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4582
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4583 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4584 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4585
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4586 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4587 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4588 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4589
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4590 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4591 return split_whitespace(self,list,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4592
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4593 else if (substring->length == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4594 return split_char(self,list,substring->str[0],maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4595
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4596 else if (substring->length == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4597 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4598 PyErr_SetString(PyExc_ValueError, "empty separator");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4599 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4600 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4601 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4602 return split_substring(self,list,substring,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4603 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4604
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4605 static
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4606 PyObject *rsplit(PyUnicodeObject *self,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4607 PyUnicodeObject *substring,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4608 int maxcount)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4609 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4610 PyObject *list;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4611
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4612 if (maxcount < 0)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4613 maxcount = INT_MAX;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4614
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4615 list = PyList_New(0);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4616 if (!list)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4617 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4618
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4619 if (substring == NULL)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4620 return rsplit_whitespace(self,list,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4621
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4622 else if (substring->length == 1)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4623 return rsplit_char(self,list,substring->str[0],maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4624
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4625 else if (substring->length == 0) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4626 Py_DECREF(list);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4627 PyErr_SetString(PyExc_ValueError, "empty separator");
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4628 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4629 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4630 else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4631 return rsplit_substring(self,list,substring,maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4632 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4633
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
4634 static
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4635 PyObject *replace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4636 PyUnicodeObject *str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4637 PyUnicodeObject *str2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4638 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4639 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4640 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4641
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4642 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4643 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4644
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4645 if (str1->length == 1 && str2->length == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4646 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4647
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4648 /* replace characters */
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4649 if (!findchar(self->str, self->length, str1->str[0]) &&
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4650 PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4651 /* nothing to replace, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4652 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4653 u = self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4654 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4655 Py_UNICODE u1 = str1->str[0];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4656 Py_UNICODE u2 = str2->str[0];
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4657
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4658 u = (PyUnicodeObject*) PyUnicode_FromUnicode(
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4659 NULL,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4660 self->length
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4661 );
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4662 if (u != NULL) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4663 Py_UNICODE_COPY(u->str, self->str,
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4664 self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4665 for (i = 0; i < u->length; i++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4666 if (u->str[i] == u1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4667 if (--maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4668 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4669 u->str[i] = u2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4670 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4671 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4672 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4673
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4674 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4675 int n, i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4676 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4677
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4678 /* replace strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4679 n = count(self, 0, self->length, str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4680 if (n > maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4681 n = maxcount;
25005
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4682 if (n == 0) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4683 /* nothing to replace, return original string */
25005
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4684 if (PyUnicode_CheckExact(self)) {
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4685 Py_INCREF(self);
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4686 u = self;
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4687 }
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4688 else {
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4689 u = (PyUnicodeObject *)
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4690 PyUnicode_FromUnicode(self->str, self->length);
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4691 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4692 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4693 u = _PyUnicode_New(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4694 self->length + n * (str2->length - str1->length));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4695 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4696 i = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4697 p = u->str;
25004
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4698 if (str1->length > 0) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4699 while (i <= self->length - str1->length)
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4700 if (Py_UNICODE_MATCH(self, i, str1)) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4701 /* replace string segment */
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4702 Py_UNICODE_COPY(p, str2->str, str2->length);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4703 p += str2->length;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4704 i += str1->length;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4705 if (--n <= 0) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4706 /* copy remaining part */
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4707 Py_UNICODE_COPY(p, self->str+i, self->length-i);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4708 break;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4709 }
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4710 } else
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4711 *p++ = self->str[i++];
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4712 } else {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4713 while (n > 0) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4714 Py_UNICODE_COPY(p, str2->str, str2->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4715 p += str2->length;
25004
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4716 if (--n <= 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4717 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4718 *p++ = self->str[i++];
25004
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4719 }
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4720 Py_UNICODE_COPY(p, self->str+i, self->length-i);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4721 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4722 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4723 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4724 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4725
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4726 return (PyObject *) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4727 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4728
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4729 /* --- Unicode Object Methods --------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4730
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4731 PyDoc_STRVAR(title__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4732 "S.title() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4733 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4734 Return a titlecased version of S, i.e. words start with title case\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4735 characters, all remaining cased characters have lower case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4736
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4737 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4738 unicode_title(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4739 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4740 return fixup(self, fixtitle);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4741 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4742
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4743 PyDoc_STRVAR(capitalize__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4744 "S.capitalize() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4745 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4746 Return a capitalized version of S, i.e. make the first character\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4747 have upper case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4748
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4749 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4750 unicode_capitalize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4751 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4752 return fixup(self, fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4753 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4754
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4755 #if 0
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4756 PyDoc_STRVAR(capwords__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4757 "S.capwords() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4758 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4759 Apply .capitalize() to all words in S and return the result with\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4760 normalized whitespace (all whitespace strings are replaced by ' ').");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4761
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4762 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4763 unicode_capwords(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4764 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4765 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4766 PyObject *item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4767 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4768
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4769 /* Split into words */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4770 list = split(self, NULL, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4771 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4772 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4773
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4774 /* Capitalize each word */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4775 for (i = 0; i < PyList_GET_SIZE(list); i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4776 item = fixup((PyUnicodeObject *)PyList_GET_ITEM(list, i),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4777 fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4778 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4779 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4780 Py_DECREF(PyList_GET_ITEM(list, i));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4781 PyList_SET_ITEM(list, i, item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4782 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4783
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4784 /* Join the words to form a new string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4785 item = PyUnicode_Join(NULL, list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4786
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4787 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4788 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4789 return (PyObject *)item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4790 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4791 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4792
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4793 /* Argument converter. Coerces to a single unicode character */
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4794
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4795 static int
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4796 convert_uc(PyObject *obj, void *addr)
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4797 {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4798 Py_UNICODE *fillcharloc = (Py_UNICODE *)addr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4799 PyObject *uniobj;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4800 Py_UNICODE *unistr;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4801
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4802 uniobj = PyUnicode_FromObject(obj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4803 if (uniobj == NULL) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4804 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4805 "The fill character cannot be converted to Unicode");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4806 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4807 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4808 if (PyUnicode_GET_SIZE(uniobj) != 1) {
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4809 PyErr_SetString(PyExc_TypeError,
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4810 "The fill character must be exactly one character long");
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4811 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4812 return 0;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4813 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4814 unistr = PyUnicode_AS_UNICODE(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4815 *fillcharloc = unistr[0];
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4816 Py_DECREF(uniobj);
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4817 return 1;
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4818 }
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4819
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4820 PyDoc_STRVAR(center__doc__,
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4821 "S.center(width[, fillchar]) -> unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4822 \n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4823 Return S centered in a Unicode string of length width. Padding is\n\
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4824 done using the specified fill character (default is a space)");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4825
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4826 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4827 unicode_center(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4828 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4829 int marg, left;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4830 int width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4831 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4832
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4833 if (!PyArg_ParseTuple(args, "i|O&:center", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4834 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4835
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4836 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4837 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4838 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4839 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4840
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4841 marg = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4842 left = marg / 2 + (marg & width & 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4843
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
4844 return (PyObject*) pad(self, left, marg - left, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4845 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4846
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4847 #if 0
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4848
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4849 /* This code should go into some future Unicode collation support
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4850 module. The basic comparison should compare ordinals on a naive
13946
7a99450da2d6 Clean up warning from Monterey compiler.
Trent Mick <trentm@activestate.com>
parents: 13890
diff changeset
4851 basis (this is what Java does and thus JPython too). */
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4852
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4853 /* speedy UTF-16 code point order comparison */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4854 /* gleaned from: */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4855 /* http://www-4.ibm.com/software/developer/library/utf16.html?dwzone=unicode */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4856
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
4857 static short utf16Fixup[32] =
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4858 {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4859 0, 0, 0, 0, 0, 0, 0, 0,
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4860 0, 0, 0, 0, 0, 0, 0, 0,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4861 0, 0, 0, 0, 0, 0, 0, 0,
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
4862 0, 0, 0, 0x2000, -0x800, -0x800, -0x800, -0x800
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4863 };
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4864
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4865 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4866 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4867 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4868 int len1, len2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4869
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4870 Py_UNICODE *s1 = str1->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4871 Py_UNICODE *s2 = str2->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4872
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4873 len1 = str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4874 len2 = str2->length;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4875
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4876 while (len1 > 0 && len2 > 0) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4877 Py_UNICODE c1, c2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4878
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4879 c1 = *s1++;
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4880 c2 = *s2++;
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4881
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4882 if (c1 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4883 c1 += utf16Fixup[c1>>11];
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4884 if (c2 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4885 c2 += utf16Fixup[c2>>11];
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4886 /* now c1 and c2 are in UTF-32-compatible order */
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4887
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4888 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4889 return (c1 < c2) ? -1 : 1;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4890
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4891 len1--; len2--;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4892 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4893
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4894 return (len1 < len2) ? -1 : (len1 != len2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4895 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4896
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4897 #else
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4898
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4899 static int
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4900 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4901 {
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4902 register int len1, len2;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4903
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4904 Py_UNICODE *s1 = str1->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4905 Py_UNICODE *s2 = str2->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4906
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4907 len1 = str1->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4908 len2 = str2->length;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4909
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4910 while (len1 > 0 && len2 > 0) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
4911 Py_UNICODE c1, c2;
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4912
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4913 c1 = *s1++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4914 c2 = *s2++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4915
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4916 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4917 return (c1 < c2) ? -1 : 1;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4918
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4919 len1--; len2--;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4920 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4921
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4922 return (len1 < len2) ? -1 : (len1 != len2);
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4923 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4924
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4925 #endif
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4926
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4927 int PyUnicode_Compare(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4928 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4929 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4930 PyUnicodeObject *u = NULL, *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4931 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4932
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4933 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4934 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4935 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4936 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4937 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4938 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4939 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4940
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13576
diff changeset
4941 /* Shortcut for empty or interned objects */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4942 if (v == u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4943 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4944 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4945 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4946 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4947
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4948 result = unicode_compare(u, v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4949
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4950 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4951 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4952 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4953
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4954 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4955 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4956 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4957 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4958 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4959
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4960 int PyUnicode_Contains(PyObject *container,
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4961 PyObject *element)
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4962 {
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4963 PyUnicodeObject *u = NULL, *v = NULL;
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4964 int result, size;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4965 register const Py_UNICODE *lhs, *end, *rhs;
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4966
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4967 /* Coerce the two arguments */
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4968 v = (PyUnicodeObject *)PyUnicode_FromObject(element);
13049
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12957
diff changeset
4969 if (v == NULL) {
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12957
diff changeset
4970 PyErr_SetString(PyExc_TypeError,
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4971 "'in <string>' requires string as left operand");
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4972 goto onError;
13049
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12957
diff changeset
4973 }
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4974 u = (PyUnicodeObject *)PyUnicode_FromObject(container);
25895
509bb0b6dfa3 Fix for bug #626172: crash using unicode latin1 single char
Marc-André Lemburg <mal@egenix.com>
parents: 25719
diff changeset
4975 if (u == NULL)
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4976 goto onError;
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4977
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4978 size = PyUnicode_GET_SIZE(v);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4979 rhs = PyUnicode_AS_UNICODE(v);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4980 lhs = PyUnicode_AS_UNICODE(u);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4981
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4982 result = 0;
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4983 if (size == 1) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4984 end = lhs + PyUnicode_GET_SIZE(u);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4985 while (lhs < end) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4986 if (*lhs++ == *rhs) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4987 result = 1;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4988 break;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4989 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4990 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4991 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4992 else {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4993 end = lhs + (PyUnicode_GET_SIZE(u) - size);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4994 while (lhs <= end) {
24608
b6707e159909 PyUnicode_Contains(): The memcmp() call didn't take into account the
Barry Warsaw <barry@python.org>
parents: 24598
diff changeset
4995 if (memcmp(lhs++, rhs, size * sizeof(Py_UNICODE)) == 0) {
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4996 result = 1;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4997 break;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4998 }
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4999 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5000 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5001
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5002 Py_DECREF(u);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5003 Py_DECREF(v);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5004 return result;
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5005
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5006 onError:
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5007 Py_XDECREF(u);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5008 Py_XDECREF(v);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5009 return -1;
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5010 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5011
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5012 /* Concat to string or Unicode object giving a new Unicode object. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5013
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5014 PyObject *PyUnicode_Concat(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5015 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5016 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5017 PyUnicodeObject *u = NULL, *v = NULL, *w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5018
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5019 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5020 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5021 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5022 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5023 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5024 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5025 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5026
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5027 /* Shortcuts */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5028 if (v == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5029 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5030 return (PyObject *)u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5031 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5032 if (u == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5033 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5034 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5035 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5036
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5037 /* Concat the two Unicode strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5038 w = _PyUnicode_New(u->length + v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5039 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5040 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5041 Py_UNICODE_COPY(w->str, u->str, u->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5042 Py_UNICODE_COPY(w->str + u->length, v->str, v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5043
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5044 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5045 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5046 return (PyObject *)w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5047
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5048 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5049 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5050 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5051 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5052 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5053
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5054 PyDoc_STRVAR(count__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5055 "S.count(sub[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5056 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5057 Return the number of occurrences of substring sub in Unicode string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5058 S[start:end]. Optional arguments start and end are\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5059 interpreted as in slice notation.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5060
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5061 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5062 unicode_count(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5063 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5064 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5065 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5066 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5067 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5068
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5069 if (!PyArg_ParseTuple(args, "O|O&O&:count", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5070 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5071 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5072
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5073 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5074 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5075 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5076 return NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5077
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5078 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5079 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5080 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5081 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5082 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5083 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5084 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5085 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5086 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5087 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5088
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5089 result = PyInt_FromLong((long) count(self, start, end, substring));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5090
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5091 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5092 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5093 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5094
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5095 PyDoc_STRVAR(encode__doc__,
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5096 "S.encode([encoding[,errors]]) -> string or unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5097 \n\
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5098 Encodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5099 to the default encoding. errors may be given to set a different error\n\
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
5100 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
5101 a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
5102 'xmlcharrefreplace' as well as any other name registered with\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
5103 codecs.register_error that can handle UnicodeEncodeErrors.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5104
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5105 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5106 unicode_encode(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5107 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5108 char *encoding = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5109 char *errors = NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5110 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5111
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5112 if (!PyArg_ParseTuple(args, "|ss:encode", &encoding, &errors))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5113 return NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5114 v = PyUnicode_AsEncodedObject((PyObject *)self, encoding, errors);
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5115 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5116 goto onError;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5117 if (!PyString_Check(v) && !PyUnicode_Check(v)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5118 PyErr_Format(PyExc_TypeError,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5119 "encoder did not return a string/unicode object "
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5120 "(type=%.400s)",
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5121 v->ob_type->tp_name);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5122 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5123 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5124 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5125 return v;
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5126
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5127 onError:
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5128 return NULL;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5129 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5130
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5131 PyDoc_STRVAR(decode__doc__,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5132 "S.decode([encoding[,errors]]) -> string or unicode\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5133 \n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5134 Decodes S using the codec registered for encoding. encoding defaults\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5135 to the default encoding. errors may be given to set a different error\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5136 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5137 a UnicodeDecodeError. Other possible values are 'ignore' and 'replace'\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5138 as well as any other name registerd with codecs.register_error that is\n\
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5139 able to handle UnicodeDecodeErrors.");
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5140
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5141 static PyObject *
32571
bec0c7618625 Fix a copy&paste typo.
Marc-André Lemburg <mal@egenix.com>
parents: 32552
diff changeset
5142 unicode_decode(PyUnicodeObject *self, PyObject *args)
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5143 {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5144 char *encoding = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5145 char *errors = NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5146 PyObject *v;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5147
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5148 if (!PyArg_ParseTuple(args, "|ss:decode", &encoding, &errors))
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5149 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5150 v = PyUnicode_AsDecodedObject((PyObject *)self, encoding, errors);
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5151 if (v == NULL)
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5152 goto onError;
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5153 if (!PyString_Check(v) && !PyUnicode_Check(v)) {
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5154 PyErr_Format(PyExc_TypeError,
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5155 "decoder did not return a string/unicode object "
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5156 "(type=%.400s)",
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5157 v->ob_type->tp_name);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5158 Py_DECREF(v);
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5159 return NULL;
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5160 }
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
5161 return v;
32552
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5162
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5163 onError:
3706a3704aea .encode()/.decode() patch part 2.
Marc-André Lemburg <mal@egenix.com>
parents: 32551
diff changeset
5164 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5165 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5166
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5167 PyDoc_STRVAR(expandtabs__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5168 "S.expandtabs([tabsize]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5169 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5170 Return a copy of S where all tab characters are expanded using spaces.\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5171 If tabsize is not given, a tab size of 8 characters is assumed.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5172
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5173 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5174 unicode_expandtabs(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5175 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5176 Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5177 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5178 Py_UNICODE *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5179 int i, j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5180 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5181 int tabsize = 8;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5182
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5183 if (!PyArg_ParseTuple(args, "|i:expandtabs", &tabsize))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5184 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5185
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13576
diff changeset
5186 /* First pass: determine size of output string */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5187 i = j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5188 e = self->str + self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5189 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5190 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5191 if (tabsize > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5192 j += tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5193 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5194 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5195 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5196 if (*p == '\n' || *p == '\r') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5197 i += j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5198 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5199 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5200 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5201
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5202 /* Second pass: create output string and fill it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5203 u = _PyUnicode_New(i + j);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5204 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5205 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5206
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5207 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5208 q = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5209
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5210 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5211 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5212 if (tabsize > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5213 i = tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5214 j += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5215 while (i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5216 *q++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5217 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5218 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5219 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5220 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5221 *q++ = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5222 if (*p == '\n' || *p == '\r')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5223 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5224 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5226 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5227 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5228
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5229 PyDoc_STRVAR(find__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5230 "S.find(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5231 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5232 Return the lowest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5233 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5234 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5235 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5236 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5237
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5238 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5239 unicode_find(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5240 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5241 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5242 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5243 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5244 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5245
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5246 if (!PyArg_ParseTuple(args, "O|O&O&:find", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5247 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5248 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5249 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5250 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5251 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5252 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5253
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5254 result = PyInt_FromLong(findstring(self, substring, start, end, 1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5255
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5256 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5257 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5258 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5259
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5260 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5261 unicode_getitem(PyUnicodeObject *self, int index)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5262 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5263 if (index < 0 || index >= self->length) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5264 PyErr_SetString(PyExc_IndexError, "string index out of range");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5265 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5266 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5267
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5268 return (PyObject*) PyUnicode_FromUnicode(&self->str[index], 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5269 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5270
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5271 static long
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5272 unicode_hash(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5273 {
13522
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5274 /* Since Unicode objects compare equal to their ASCII string
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5275 counterparts, they should use the individual character values
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5276 as basis for their hash value. This is needed to assure that
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5277 strings and Unicode objects behave in the same way as
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5278 dictionary keys. */
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5279
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5280 register int len;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5281 register Py_UNICODE *p;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5282 register long x;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5283
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5284 if (self->hash != -1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5285 return self->hash;
13522
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5286 len = PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5287 p = PyUnicode_AS_UNICODE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5288 x = *p << 7;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5289 while (--len >= 0)
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5290 x = (1000003*x) ^ *p++;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5291 x ^= PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5292 if (x == -1)
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5293 x = -2;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5294 self->hash = x;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
5295 return x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5296 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5297
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5298 PyDoc_STRVAR(index__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5299 "S.index(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5300 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5301 Like S.find() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5302
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5303 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5304 unicode_index(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5305 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5306 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5307 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5308 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5309 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5310
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5311 if (!PyArg_ParseTuple(args, "O|O&O&:index", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5312 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5313 return NULL;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5314
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5315 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5316 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5317 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5318 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5319
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5320 result = findstring(self, substring, start, end, 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5321
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5322 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5323 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5324 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5325 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5326 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5327 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5328 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5329
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5330 PyDoc_STRVAR(islower__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5331 "S.islower() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5332 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5333 Return True if all cased characters in S are lowercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5334 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5335
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5336 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5337 unicode_islower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5338 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5339 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5340 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5341 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5342
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5343 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5344 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5345 return PyBool_FromLong(Py_UNICODE_ISLOWER(*p));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5346
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5347 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5348 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5349 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5350
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5351 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5352 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5353 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5354 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5355
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5356 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5357 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5358 else if (!cased && Py_UNICODE_ISLOWER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5359 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5360 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5361 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5362 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5363
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5364 PyDoc_STRVAR(isupper__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5365 "S.isupper() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5366 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5367 Return True if all cased characters in S are uppercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5368 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5369
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5370 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5371 unicode_isupper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5372 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5373 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5374 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5375 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5376
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5377 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5378 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5379 return PyBool_FromLong(Py_UNICODE_ISUPPER(*p) != 0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5380
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5381 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5382 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5383 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5384
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5385 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5386 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5387 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5388 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5389
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5390 if (Py_UNICODE_ISLOWER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5391 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5392 else if (!cased && Py_UNICODE_ISUPPER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5393 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5394 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5395 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5396 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5397
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5398 PyDoc_STRVAR(istitle__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5399 "S.istitle() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5400 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5401 Return True if S is a titlecased string and there is at least one\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5402 character in S, i.e. upper- and titlecase characters may only\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5403 follow uncased characters and lowercase characters only cased ones.\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5404 Return False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5405
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5406 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5407 unicode_istitle(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5408 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5409 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5410 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5411 int cased, previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5412
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5413 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5414 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5415 return PyBool_FromLong((Py_UNICODE_ISTITLE(*p) != 0) ||
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5416 (Py_UNICODE_ISUPPER(*p) != 0));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5417
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5418 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5419 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5420 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5421
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5422 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5423 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5424 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5425 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5426 register const Py_UNICODE ch = *p;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5427
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5428 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5429 if (previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5430 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5431 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5432 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5433 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5434 else if (Py_UNICODE_ISLOWER(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5435 if (!previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5436 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5437 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5438 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5439 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5440 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5441 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5442 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5443 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5444 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5445
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5446 PyDoc_STRVAR(isspace__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5447 "S.isspace() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5448 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5449 Return True if all characters in S are whitespace\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5450 and there is at least one character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5451
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5452 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5453 unicode_isspace(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5454 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5455 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5456 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5457
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5458 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5459 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5460 Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5461 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5462
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5463 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5464 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5465 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5466
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5467 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5468 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5469 if (!Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5470 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5471 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5472 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5473 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5474
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5475 PyDoc_STRVAR(isalpha__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5476 "S.isalpha() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5477 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5478 Return True if all characters in S are alphabetic\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5479 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5480
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5481 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5482 unicode_isalpha(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5483 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5484 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5485 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5486
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5487 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5488 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5489 Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5490 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5491
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5492 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5493 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5494 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5495
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5496 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5497 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5498 if (!Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5499 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5500 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5501 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5502 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5503
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5504 PyDoc_STRVAR(isalnum__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5505 "S.isalnum() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5506 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5507 Return True if all characters in S are alphanumeric\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5508 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5509
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5510 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5511 unicode_isalnum(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5512 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5513 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5514 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5515
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5516 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5517 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5518 Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5519 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5520
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5521 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5522 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5523 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5524
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5525 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5526 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5527 if (!Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5528 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5529 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5530 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5531 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
5532
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5533 PyDoc_STRVAR(isdecimal__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5534 "S.isdecimal() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5535 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5536 Return True if there are only decimal characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5537 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5538
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5539 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5540 unicode_isdecimal(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5541 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5542 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5543 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5544
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5545 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5546 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5547 Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5548 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5549
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5550 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5551 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5552 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5553
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5554 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5555 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5556 if (!Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5557 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5558 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5559 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5560 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5561
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5562 PyDoc_STRVAR(isdigit__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5563 "S.isdigit() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5564 \n\
30707
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5565 Return True if all characters in S are digits\n\
9e21e56bf5f3 Patch #825679: Clarify semantics of .isfoo on empty strings.
Martin v. Löwis <martin@v.loewis.de>
parents: 30653
diff changeset
5566 and there is at least one character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5567
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5568 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5569 unicode_isdigit(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5570 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5571 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5572 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5573
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5574 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5575 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5576 Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5577 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5578
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5579 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5580 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5581 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5582
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5583 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5584 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5585 if (!Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5586 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5587 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5588 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5589 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5590
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5591 PyDoc_STRVAR(isnumeric__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5592 "S.isnumeric() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5593 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5594 Return True if there are only numeric characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5595 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5596
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5597 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5598 unicode_isnumeric(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5599 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5600 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5601 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5602
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5603 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5604 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5605 Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5606 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5607
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5608 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5609 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5610 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5611
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5612 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5613 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5614 if (!Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5615 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5616 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5617 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5618 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5619
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5620 PyDoc_STRVAR(join__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5621 "S.join(sequence) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5622 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5623 Return a string which is the concatenation of the strings in the\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5624 sequence. The separator between elements is S.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5625
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5626 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5627 unicode_join(PyObject *self, PyObject *data)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5628 {
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5629 return PyUnicode_Join(self, data);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5630 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5631
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5632 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5633 unicode_length(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5634 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5635 return self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5636 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5637
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5638 PyDoc_STRVAR(ljust__doc__,
32197
3a84e23b042f - SF #962502: Add two more methods for unicode type; width() and
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31873
diff changeset
5639 "S.ljust(width[, fillchar]) -> int\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5640 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5641 Return S left justified in a Unicode string of length width. Padding is\n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5642 done using the specified fill character (default is a space).");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5643
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5644 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5645 unicode_ljust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5646 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5647 int width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5648 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5649
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5650 if (!PyArg_ParseTuple(args, "i|O&:ljust", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5651 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5652
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5653 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5654 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5655 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5656 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5657
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
5658 return (PyObject*) pad(self, 0, width - self->length, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5659 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5660
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5661 PyDoc_STRVAR(lower__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5662 "S.lower() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5663 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5664 Return a copy of the string S converted to lowercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5665
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5666 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5667 unicode_lower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5668 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5669 return fixup(self, fixlower);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5670 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5671
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5672 #define LEFTSTRIP 0
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5673 #define RIGHTSTRIP 1
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5674 #define BOTHSTRIP 2
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5675
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5676 /* Arrays indexed by above */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5677 static const char *stripformat[] = {"|O:lstrip", "|O:rstrip", "|O:strip"};
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5678
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5679 #define STRIPNAME(i) (stripformat[i]+3)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5680
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5681 static const Py_UNICODE *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5682 unicode_memchr(const Py_UNICODE *s, Py_UNICODE c, size_t n)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5683 {
23228
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which
Tim Peters <tim.peters@gmail.com>
parents: 23225
diff changeset
5684 size_t i;
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which
Tim Peters <tim.peters@gmail.com>
parents: 23225
diff changeset
5685 for (i = 0; i < n; ++i)
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which
Tim Peters <tim.peters@gmail.com>
parents: 23225
diff changeset
5686 if (s[i] == c)
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5687 return s+i;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5688 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5689 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5690
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5691 /* externally visible for str.strip(unicode) */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5692 PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5693 _PyUnicode_XStrip(PyUnicodeObject *self, int striptype, PyObject *sepobj)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5694 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5695 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5696 int len = PyUnicode_GET_SIZE(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5697 Py_UNICODE *sep = PyUnicode_AS_UNICODE(sepobj);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5698 int seplen = PyUnicode_GET_SIZE(sepobj);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5699 int i, j;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5700
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5701 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5702 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5703 while (i < len && unicode_memchr(sep, s[i], seplen)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5704 i++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5705 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5706 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5707
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5708 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5709 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5710 do {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5711 j--;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5712 } while (j >= i && unicode_memchr(sep, s[j], seplen));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5713 j++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5714 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5715
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5716 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5717 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5718 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5719 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5720 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5721 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5722 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5723
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5724
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5725 static PyObject *
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5726 do_strip(PyUnicodeObject *self, int striptype)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5727 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5728 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5729 int len = PyUnicode_GET_SIZE(self), i, j;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5730
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5731 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5732 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5733 while (i < len && Py_UNICODE_ISSPACE(s[i])) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5734 i++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5735 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5736 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5737
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5738 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5739 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5740 do {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5741 j--;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5742 } while (j >= i && Py_UNICODE_ISSPACE(s[j]));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5743 j++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5744 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5745
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5746 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5747 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5748 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5749 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5750 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5751 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5752 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5753
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5754
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5755 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5756 do_argstrip(PyUnicodeObject *self, int striptype, PyObject *args)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5757 {
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5758 PyObject *sep = NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5759
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5760 if (!PyArg_ParseTuple(args, (char *)stripformat[striptype], &sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5761 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5762
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5763 if (sep != NULL && sep != Py_None) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5764 if (PyUnicode_Check(sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5765 return _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5766 else if (PyString_Check(sep)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5767 PyObject *res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5768 sep = PyUnicode_FromObject(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5769 if (sep==NULL)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5770 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5771 res = _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5772 Py_DECREF(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5773 return res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5774 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5775 else {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5776 PyErr_Format(PyExc_TypeError,
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5777 "%s arg must be None, unicode or str",
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5778 STRIPNAME(striptype));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5779 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5780 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5781 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5782
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5783 return do_strip(self, striptype);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5784 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5785
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5786
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5787 PyDoc_STRVAR(strip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5788 "S.strip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5789 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5790 Return a copy of the string S with leading and trailing\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5791 whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5792 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5793 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5794
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5795 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5796 unicode_strip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5797 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5798 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5799 return do_strip(self, BOTHSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5800 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5801 return do_argstrip(self, BOTHSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5802 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5803
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5804
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5805 PyDoc_STRVAR(lstrip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5806 "S.lstrip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5807 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5808 Return a copy of the string S with leading whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5809 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5810 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5811
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5812 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5813 unicode_lstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5814 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5815 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5816 return do_strip(self, LEFTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5817 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5818 return do_argstrip(self, LEFTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5819 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5820
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5821
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5822 PyDoc_STRVAR(rstrip__doc__,
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5823 "S.rstrip([chars]) -> unicode\n\
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5824 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5825 Return a copy of the string S with trailing whitespace removed.\n\
28575
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5826 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string *strip methods the same.
Neal Norwitz <nnorwitz@gmail.com>
parents: 28559
diff changeset
5827 If chars is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5828
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5829 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5830 unicode_rstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5831 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5832 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5833 return do_strip(self, RIGHTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5834 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5835 return do_argstrip(self, RIGHTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5836 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5837
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5838
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5839 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5840 unicode_repeat(PyUnicodeObject *str, int len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5841 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5842 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5843 Py_UNICODE *p;
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5844 int nchars;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5845 size_t nbytes;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5846
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5847 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5848 len = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5849
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5850 if (len == 1 && PyUnicode_CheckExact(str)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5851 /* no repeat, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5852 Py_INCREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5853 return (PyObject*) str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5854 }
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5855
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5856 /* ensure # of chars needed doesn't overflow int and # of bytes
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5857 * needed doesn't overflow size_t
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5858 */
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5859 nchars = len * str->length;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5860 if (len && nchars / len != str->length) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5861 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5862 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5863 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5864 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5865 nbytes = (nchars + 1) * sizeof(Py_UNICODE);
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5866 if (nbytes / sizeof(Py_UNICODE) != (size_t)(nchars + 1)) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5867 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5868 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5869 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5870 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5871 u = _PyUnicode_New(nchars);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5872 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5873 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5874
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5875 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5876
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5877 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5878 Py_UNICODE_COPY(p, str->str, str->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5879 p += str->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5880 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5881
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5882 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5883 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5884
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5885 PyObject *PyUnicode_Replace(PyObject *obj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5886 PyObject *subobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5887 PyObject *replobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5888 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5889 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5890 PyObject *self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5891 PyObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5892 PyObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5893 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5894
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5895 self = PyUnicode_FromObject(obj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5896 if (self == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5897 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5898 str1 = PyUnicode_FromObject(subobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5899 if (str1 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5900 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5901 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5902 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5903 str2 = PyUnicode_FromObject(replobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5904 if (str2 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5905 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5906 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5907 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5908 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5909 result = replace((PyUnicodeObject *)self,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5910 (PyUnicodeObject *)str1,
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
5911 (PyUnicodeObject *)str2,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5912 maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5913 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5914 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5915 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5916 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5917 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5918
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5919 PyDoc_STRVAR(replace__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5920 "S.replace (old, new[, maxsplit]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5921 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5922 Return a copy of S with all occurrences of substring\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5923 old replaced by new. If the optional argument maxsplit is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5924 given, only the first maxsplit occurrences are replaced.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5925
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5926 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5927 unicode_replace(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5928 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5929 PyUnicodeObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5930 PyUnicodeObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5931 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5932 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5933
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5934 if (!PyArg_ParseTuple(args, "OO|i:replace", &str1, &str2, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5935 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5936 str1 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5937 if (str1 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5938 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5939 str2 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str2);
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
5940 if (str2 == NULL) {
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
5941 Py_DECREF(str1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5942 return NULL;
27774
bfc524379307 Fix two refcounting bugs
Walter Dörwald <walter@livinglogic.de>
parents: 27555
diff changeset
5943 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5944
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5945 result = replace(self, str1, str2, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5946
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5947 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5948 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5949 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5950 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5951
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5952 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5953 PyObject *unicode_repr(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5954 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5955 return unicodeescape_string(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5956 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5957 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5958 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5959
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5960 PyDoc_STRVAR(rfind__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5961 "S.rfind(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5962 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5963 Return the highest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5964 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5965 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5966 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5967 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5969 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5970 unicode_rfind(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5971 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5972 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5973 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5974 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5975 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5976
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5977 if (!PyArg_ParseTuple(args, "O|O&O&:rfind", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5978 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5979 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5980 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5981 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5982 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5983 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5984
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5985 result = PyInt_FromLong(findstring(self, substring, start, end, -1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5986
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5987 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5988 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5989 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5990
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5991 PyDoc_STRVAR(rindex__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5992 "S.rindex(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5993 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5994 Like S.rfind() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5995
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5996 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5997 unicode_rindex(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5998 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5999 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6000 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6001 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6002 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6003
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6004 if (!PyArg_ParseTuple(args, "O|O&O&:rindex", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6005 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6006 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6007 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6008 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6009 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6010 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6011
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6012 result = findstring(self, substring, start, end, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6013
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6014 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6015 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6016 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6017 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6018 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6019 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6020 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6021
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6022 PyDoc_STRVAR(rjust__doc__,
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6023 "S.rjust(width[, fillchar]) -> unicode\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6024 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6025 Return S right justified in a Unicode string of length width. Padding is\n\
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6026 done using the specified fill character (default is a space).");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6027
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6028 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6029 unicode_rjust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6030 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6031 int width;
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6032 Py_UNICODE fillchar = ' ';
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6033
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6034 if (!PyArg_ParseTuple(args, "i|O&:rjust", &width, convert_uc, &fillchar))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6035 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6036
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
6037 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6038 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6039 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6040 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6041
31021
21a10c3600b5 Add optional fillchar argument to ljust(), rjust(), and center() string methods.
Raymond Hettinger <python@rcn.com>
parents: 30765
diff changeset
6042 return (PyObject*) pad(self, width - self->length, 0, fillchar);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6043 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6044
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6045 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6046 unicode_slice(PyUnicodeObject *self, int start, int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6047 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6048 /* standard clamping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6049 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6050 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6051 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6052 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6053 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6054 end = self->length;
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
6055 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6056 /* full slice, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6057 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6058 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6059 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6060 if (start > end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6061 start = end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6062 /* copy slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6063 return (PyObject*) PyUnicode_FromUnicode(self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6064 end - start);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6065 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6066
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6067 PyObject *PyUnicode_Split(PyObject *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6068 PyObject *sep,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6069 int maxsplit)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6070 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6071 PyObject *result;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6072
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6073 s = PyUnicode_FromObject(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6074 if (s == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6075 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6076 if (sep != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6077 sep = PyUnicode_FromObject(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6078 if (sep == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6079 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6080 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6081 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6082 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6083
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6084 result = split((PyUnicodeObject *)s, (PyUnicodeObject *)sep, maxsplit);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6085
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6086 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6087 Py_XDECREF(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6088 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6089 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6090
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6091 PyDoc_STRVAR(split__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6092 "S.split([sep [,maxsplit]]) -> list of strings\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6093 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6094 Return a list of the words in S, using sep as the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6095 delimiter string. If maxsplit is given, at most maxsplit\n\
33495
2d6e1854082e Add a missing line continuation character.
Thomas Heller <theller@ctypes.org>
parents: 33487
diff changeset
6096 splits are done. If sep is not specified or is None,\n\
33486
53c1357c9c63 Enhance the docstrings for unicode.split() and string.split()
Walter Dörwald <walter@livinglogic.de>
parents: 33428
diff changeset
6097 any whitespace string is a separator.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6098
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6099 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6100 unicode_split(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6101 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6102 PyObject *substring = Py_None;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6103 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6104
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6105 if (!PyArg_ParseTuple(args, "|Oi:split", &substring, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6106 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6107
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6108 if (substring == Py_None)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6109 return split(self, NULL, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6110 else if (PyUnicode_Check(substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6111 return split(self, (PyUnicodeObject *)substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6112 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6113 return PyUnicode_Split((PyObject *)self, substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6114 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6115
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6116 PyObject *PyUnicode_RSplit(PyObject *s,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6117 PyObject *sep,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6118 int maxsplit)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6119 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6120 PyObject *result;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6121
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6122 s = PyUnicode_FromObject(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6123 if (s == NULL)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6124 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6125 if (sep != NULL) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6126 sep = PyUnicode_FromObject(sep);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6127 if (sep == NULL) {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6128 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6129 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6130 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6131 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6132
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6133 result = rsplit((PyUnicodeObject *)s, (PyUnicodeObject *)sep, maxsplit);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6134
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6135 Py_DECREF(s);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6136 Py_XDECREF(sep);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6137 return result;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6138 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6139
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6140 PyDoc_STRVAR(rsplit__doc__,
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6141 "S.rsplit([sep [,maxsplit]]) -> list of strings\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6142 \n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6143 Return a list of the words in S, using sep as the\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6144 delimiter string, starting at the end of the string and\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6145 working to the front. If maxsplit is given, at most maxsplit\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6146 splits are done. If sep is not specified, any whitespace string\n\
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6147 is a separator.");
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6148
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6149 static PyObject*
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6150 unicode_rsplit(PyUnicodeObject *self, PyObject *args)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6151 {
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6152 PyObject *substring = Py_None;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6153 int maxcount = -1;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6154
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6155 if (!PyArg_ParseTuple(args, "|Oi:rsplit", &substring, &maxcount))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6156 return NULL;
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6157
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6158 if (substring == Py_None)
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6159 return rsplit(self, NULL, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6160 else if (PyUnicode_Check(substring))
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6161 return rsplit(self, (PyUnicodeObject *)substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6162 else
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6163 return PyUnicode_RSplit((PyObject *)self, substring, maxcount);
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6164 }
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6165
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6166 PyDoc_STRVAR(splitlines__doc__,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
6167 "S.splitlines([keepends]]) -> list of strings\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6168 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6169 Return a list of the lines in S, breaking at line boundaries.\n\
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
6170 Line breaks are not included in the resulting list unless keepends\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6171 is given and true.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6172
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6173 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6174 unicode_splitlines(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6175 {
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
6176 int keepends = 0;
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
6177
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
6178 if (!PyArg_ParseTuple(args, "|i:splitlines", &keepends))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6179 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6180
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
6181 return PyUnicode_Splitlines((PyObject *)self, keepends);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6182 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6183
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6184 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6185 PyObject *unicode_str(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6186 {
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6187 return PyUnicode_AsEncodedString((PyObject *)self, NULL, NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6188 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6189
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6190 PyDoc_STRVAR(swapcase__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6191 "S.swapcase() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6192 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6193 Return a copy of S with uppercase characters converted to lowercase\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6194 and vice versa.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6195
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6196 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6197 unicode_swapcase(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6198 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6199 return fixup(self, fixswapcase);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6200 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6201
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6202 PyDoc_STRVAR(translate__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6203 "S.translate(table) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6204 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6205 Return a copy of the string S, where all characters have been mapped\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6206 through the given translation table, which must be a mapping of\n\
25120
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
6207 Unicode ordinals to Unicode ordinals, Unicode strings or None.\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
6208 Unmapped characters are left untouched. Characters mapped to None\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
6209 are deleted.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6210
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6211 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6212 unicode_translate(PyUnicodeObject *self, PyObject *table)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6213 {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6214 return PyUnicode_TranslateCharmap(self->str,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6215 self->length,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6216 table,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6217 "ignore");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6218 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6219
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6220 PyDoc_STRVAR(upper__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6221 "S.upper() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6222 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6223 Return a copy of S converted to uppercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6224
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6225 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6226 unicode_upper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6227 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6228 return fixup(self, fixupper);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6229 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6230
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6231 PyDoc_STRVAR(zfill__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6232 "S.zfill(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6233 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6234 Pad a numeric string x with zeros on the left, to fill a field\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6235 of the specified width. The string x is never truncated.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6236
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6237 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6238 unicode_zfill(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6239 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6240 int fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6241 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6242
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6243 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6244 if (!PyArg_ParseTuple(args, "i:zfill", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6245 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6246
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6247 if (self->length >= width) {
23051
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6248 if (PyUnicode_CheckExact(self)) {
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6249 Py_INCREF(self);
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6250 return (PyObject*) self;
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6251 }
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6252 else
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6253 return PyUnicode_FromUnicode(
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6254 PyUnicode_AS_UNICODE(self),
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6255 PyUnicode_GET_SIZE(self)
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
6256 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6257 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6258
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6259 fill = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6260
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6261 u = pad(self, fill, 0, '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6262
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
6263 if (u == NULL)
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
6264 return NULL;
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
6265
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6266 if (u->str[fill] == '+' || u->str[fill] == '-') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6267 /* move sign to beginning of string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6268 u->str[0] = u->str[fill];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6269 u->str[fill] = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6270 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6271
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6272 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6273 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6274
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6275 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6276 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6277 unicode_freelistsize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6278 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6279 return PyInt_FromLong(unicode_freelist_size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6280 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6281 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6282
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6283 PyDoc_STRVAR(startswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6284 "S.startswith(prefix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6285 \n\
28559
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
6286 Return True if S starts with the specified prefix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
6287 With optional start, test S beginning at that position.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
6288 With optional end, stop comparing S at that position.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6289
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6290 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6291 unicode_startswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6292 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6293 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6294 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6295 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6296 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6297 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6298
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6299 if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6300 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6301 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6302 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6303 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6304 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6305 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6306
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6307 result = PyBool_FromLong(tailmatch(self, substring, start, end, -1));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6308
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6309 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6310 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6311 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6312
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6313
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6314 PyDoc_STRVAR(endswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6315 "S.endswith(suffix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6316 \n\
28559
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
6317 Return True if S ends with the specified suffix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
6318 With optional start, test S beginning at that position.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output.
Guido van Rossum <guido@python.org>
parents: 28494
diff changeset
6319 With optional end, stop comparing S at that position.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6320
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6321 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6322 unicode_endswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6323 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6324 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6325 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6326 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6327 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6328 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6329
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6330 if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
6331 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6332 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6333 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6334 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6335 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6336 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6337
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
6338 result = PyBool_FromLong(tailmatch(self, substring, start, end, +1));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6340 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6341 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6342 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6343
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6344
27521
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6345
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6346 static PyObject *
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6347 unicode_getnewargs(PyUnicodeObject *v)
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6348 {
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6349 return Py_BuildValue("(u#)", v->str, v->length);
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6350 }
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6351
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6352
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6353 static PyMethodDef unicode_methods[] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6354
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6355 /* Order is according to common usage: often used methods should
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6356 appear first, since lookup is done sequentially. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6357
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6358 {"encode", (PyCFunction) unicode_encode, METH_VARARGS, encode__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6359 {"replace", (PyCFunction) unicode_replace, METH_VARARGS, replace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6360 {"split", (PyCFunction) unicode_split, METH_VARARGS, split__doc__},
31199
504943c89193 Add rsplit method for str and unicode builtin types.
Hye-Shik Chang <hyeshik@gmail.com>
parents: 31042
diff changeset
6361 {"rsplit", (PyCFunction) unicode_rsplit, METH_VARARGS, rsplit__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6362 {"join", (PyCFunction) unicode_join, METH_O, join__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6363 {"capitalize", (PyCFunction) unicode_capitalize, METH_NOARGS, capitalize__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6364 {"title", (PyCFunction) unicode_title, METH_NOARGS, title__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6365 {"center", (PyCFunction) unicode_center, METH_VARARGS, center__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6366 {"count", (PyCFunction) unicode_count, METH_VARARGS, count__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6367 {"expandtabs", (PyCFunction) unicode_expandtabs, METH_VARARGS, expandtabs__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6368 {"find", (PyCFunction) unicode_find, METH_VARARGS, find__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6369 {"index", (PyCFunction) unicode_index, METH_VARARGS, index__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6370 {"ljust", (PyCFunction) unicode_ljust, METH_VARARGS, ljust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6371 {"lower", (PyCFunction) unicode_lower, METH_NOARGS, lower__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6372 {"lstrip", (PyCFunction) unicode_lstrip, METH_VARARGS, lstrip__doc__},
32551
f46d49e2e0f0 Allow string and unicode return types from .encode()/.decode()
Marc-André Lemburg <mal@egenix.com>
parents: 32363
diff changeset
6373 {"decode", (PyCFunction) unicode_decode, METH_VARARGS, decode__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6374 /* {"maketrans", (PyCFunction) unicode_maketrans, METH_VARARGS, maketrans__doc__}, */
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6375 {"rfind", (PyCFunction) unicode_rfind, METH_VARARGS, rfind__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6376 {"rindex", (PyCFunction) unicode_rindex, METH_VARARGS, rindex__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6377 {"rjust", (PyCFunction) unicode_rjust, METH_VARARGS, rjust__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6378 {"rstrip", (PyCFunction) unicode_rstrip, METH_VARARGS, rstrip__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6379 {"splitlines", (PyCFunction) unicode_splitlines, METH_VARARGS, splitlines__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
6380 {"strip", (PyCFunction) unicode_strip, METH_VARARGS, strip__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6381 {"swapcase", (PyCFunction) unicode_swapcase, METH_NOARGS, swapcase__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6382 {"translate", (PyCFunction) unicode_translate, METH_O, translate__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6383 {"upper", (PyCFunction) unicode_upper, METH_NOARGS, upper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6384 {"startswith", (PyCFunction) unicode_startswith, METH_VARARGS, startswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6385 {"endswith", (PyCFunction) unicode_endswith, METH_VARARGS, endswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6386 {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6387 {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6388 {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6389 {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6390 {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6391 {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6392 {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6393 {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6394 {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
6395 {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6396 #if 0
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6397 {"capwords", (PyCFunction) unicode_capwords, METH_NOARGS, capwords__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6398 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6399
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6400 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6401 /* This one is just used for debugging the implementation. */
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
6402 {"freelistsize", (PyCFunction) unicode_freelistsize, METH_NOARGS},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6403 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6404
27521
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin
Guido van Rossum <guido@python.org>
parents: 27175
diff changeset
6405 {"__getnewargs__", (PyCFunction)unicode_getnewargs, METH_NOARGS},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6406 {NULL, NULL}
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6407 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6408
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6409 static PyObject *
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6410 unicode_mod(PyObject *v, PyObject *w)
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6411 {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6412 if (!PyUnicode_Check(v)) {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6413 Py_INCREF(Py_NotImplemented);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6414 return Py_NotImplemented;
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6415 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6416 return PyUnicode_Format(v, w);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6417 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6418
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6419 static PyNumberMethods unicode_as_number = {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6420 0, /*nb_add*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6421 0, /*nb_subtract*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6422 0, /*nb_multiply*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6423 0, /*nb_divide*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6424 unicode_mod, /*nb_remainder*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6425 };
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6426
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6427 static PySequenceMethods unicode_as_sequence = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6428 (inquiry) unicode_length, /* sq_length */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6429 (binaryfunc) PyUnicode_Concat, /* sq_concat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6430 (intargfunc) unicode_repeat, /* sq_repeat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6431 (intargfunc) unicode_getitem, /* sq_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6432 (intintargfunc) unicode_slice, /* sq_slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6433 0, /* sq_ass_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6434 0, /* sq_ass_slice */
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
6435 (objobjproc)PyUnicode_Contains, /*sq_contains*/
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6436 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6437
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6438 static PyObject*
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6439 unicode_subscript(PyUnicodeObject* self, PyObject* item)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6440 {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6441 if (PyInt_Check(item)) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6442 long i = PyInt_AS_LONG(item);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6443 if (i < 0)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6444 i += PyString_GET_SIZE(self);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6445 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6446 } else if (PyLong_Check(item)) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6447 long i = PyLong_AsLong(item);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6448 if (i == -1 && PyErr_Occurred())
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6449 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6450 if (i < 0)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6451 i += PyString_GET_SIZE(self);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6452 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6453 } else if (PySlice_Check(item)) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6454 int start, stop, step, slicelength, cur, i;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6455 Py_UNICODE* source_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6456 Py_UNICODE* result_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6457 PyObject* result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6458
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6459 if (PySlice_GetIndicesEx((PySliceObject*)item, PyString_GET_SIZE(self),
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6460 &start, &stop, &step, &slicelength) < 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6461 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6462 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6463
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6464 if (slicelength <= 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6465 return PyUnicode_FromUnicode(NULL, 0);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6466 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6467 source_buf = PyUnicode_AS_UNICODE((PyObject*)self);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6468 result_buf = PyMem_MALLOC(slicelength*sizeof(Py_UNICODE));
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6469
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6470 for (cur = start, i = 0; i < slicelength; cur += step, i++) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6471 result_buf[i] = source_buf[cur];
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6472 }
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6473
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6474 result = PyUnicode_FromUnicode(result_buf, slicelength);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6475 PyMem_FREE(result_buf);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6476 return result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6477 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6478 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6479 PyErr_SetString(PyExc_TypeError, "string indices must be integers");
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6480 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6481 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6482 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6483
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6484 static PyMappingMethods unicode_as_mapping = {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6485 (inquiry)unicode_length, /* mp_length */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6486 (binaryfunc)unicode_subscript, /* mp_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6487 (objobjargproc)0, /* mp_ass_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6488 };
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6489
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6490 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6491 unicode_buffer_getreadbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6492 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6493 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6494 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6495 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6496 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6497 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6498 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6499 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6500 *ptr = (void *) self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6501 return PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6502 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6503
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6504 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6505 unicode_buffer_getwritebuf(PyUnicodeObject *self, int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6506 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6507 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6508 PyErr_SetString(PyExc_TypeError,
23837
359a57e96d5e Fix typo in exception message
Neal Norwitz <nnorwitz@gmail.com>
parents: 23832
diff changeset
6509 "cannot use unicode as modifiable buffer");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6510 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6511 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6512
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6513 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6514 unicode_buffer_getsegcount(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6515 int *lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6516 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6517 if (lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6518 *lenp = PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6519 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6520 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6521
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6522 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6523 unicode_buffer_getcharbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6524 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6525 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6526 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6527 PyObject *str;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6528
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6529 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6530 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6531 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6532 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6533 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
6534 str = _PyUnicode_AsDefaultEncodedString((PyObject *)self, NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6535 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6536 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6537 *ptr = (void *) PyString_AS_STRING(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6538 return PyString_GET_SIZE(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6539 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6540
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6541 /* Helpers for PyUnicode_Format() */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6542
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6543 static PyObject *
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
6544 getnextarg(PyObject *args, int arglen, int *p_argidx)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6545 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6546 int argidx = *p_argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6547 if (argidx < arglen) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6548 (*p_argidx)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6549 if (arglen < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6550 return args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6551 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6552 return PyTuple_GetItem(args, argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6553 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6554 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6555 "not enough arguments for format string");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6556 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6557 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6558
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6559 #define F_LJUST (1<<0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6560 #define F_SIGN (1<<1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6561 #define F_BLANK (1<<2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6562 #define F_ALT (1<<3)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6563 #define F_ZERO (1<<4)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6564
36262
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6565 static int
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6566 strtounicode(Py_UNICODE *buffer, const char *charbuffer)
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6567 {
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6568 register long i;
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6569 long len = strlen(charbuffer);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6570 for (i = len - 1; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6571 buffer[i] = (Py_UNICODE) charbuffer[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6572
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6573 return len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6574 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6575
36262
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6576 static int
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6577 doubletounicode(Py_UNICODE *buffer, size_t len, const char *format, double x)
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6578 {
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6579 PyOS_ascii_formatd((char *)buffer, len, format, x);
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6580 return strtounicode(buffer, (char *)buffer);
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6581 }
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6582
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6583 static int
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6584 longtounicode(Py_UNICODE *buffer, size_t len, const char *format, long x)
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6585 {
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6586 PyOS_snprintf((char *)buffer, len, format, x);
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6587 return strtounicode(buffer, (char *)buffer);
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6588 }
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6589
24736
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6590 /* XXX To save some code duplication, formatfloat/long/int could have been
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6591 shared with stringobject.c, converting from 8-bit to Unicode after the
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6592 formatting is done. */
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6593
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6594 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6595 formatfloat(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6596 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6597 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6598 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6599 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6600 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6601 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6602 /* fmt = '%#.' + `prec` + `type`
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6603 worst case length = 3 + 10 (len of INT_MAX) + 1 = 14 (use 20)*/
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6604 char fmt[20];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6605 double x;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6606
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6607 x = PyFloat_AsDouble(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6608 if (x == -1.0 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6609 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6610 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6611 prec = 6;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6612 if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6613 type = 'g';
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6614 /* Worst case length calc to ensure no buffer overrun:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6615
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6616 'g' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6617 fmt = %#.<prec>g
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6618 buf = '-' + [0-9]*prec + '.' + 'e+' + (longest exp
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6619 for any double rep.)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6620 len = 1 + prec + 1 + 2 + 5 = 9 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6621
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6622 'f' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6623 buf = '-' + [0-9]*x + '.' + [0-9]*prec (with x < 50)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6624 len = 1 + 50 + 1 + prec = 52 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6625
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6626 If prec=0 the effective precision is 1 (the leading digit is
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6627 always given), therefore increase the length by one.
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6628
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6629 */
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6630 if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) ||
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6631 (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) {
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6632 PyErr_SetString(PyExc_OverflowError,
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6633 "formatted float is too long (precision too large?)");
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6634 return -1;
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6635 }
21082
c8d775b9ff92 formatfloat(), formatint(): Conversion of sprintf() to PyOS_snprintf()
Barry Warsaw <barry@python.org>
parents: 21067
diff changeset
6636 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%d%c",
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6637 (flags&F_ALT) ? "#" : "",
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6638 prec, type);
36262
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6639 return doubletounicode(buf, buflen, fmt, x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6640 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6641
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6642 static PyObject*
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6643 formatlong(PyObject *val, int flags, int prec, int type)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6644 {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6645 char *buf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6646 int i, len;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6647 PyObject *str; /* temporary string object. */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6648 PyUnicodeObject *result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6649
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6650 str = _PyString_FormatLong(val, flags, prec, type, &buf, &len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6651 if (!str)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6652 return NULL;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6653 result = _PyUnicode_New(len);
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
6654 if (!result) {
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
6655 Py_DECREF(str);
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
6656 return NULL;
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
6657 }
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6658 for (i = 0; i < len; i++)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6659 result->str[i] = buf[i];
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6660 result->str[len] = 0;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6661 Py_DECREF(str);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6662 return (PyObject*)result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6663 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6664
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6665 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6666 formatint(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6667 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6668 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6669 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6670 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6671 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6672 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6673 /* fmt = '%#.' + `prec` + 'l' + `type`
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6674 * worst case length = 3 + 19 (worst len of INT_MAX on 64-bit machine)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6675 * + 1 + 1
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6676 * = 24
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6677 */
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6678 char fmt[64]; /* plenty big enough! */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6679 char *sign;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6680 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6681
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6682 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6683 if (x == -1 && PyErr_Occurred())
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6684 return -1;
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6685 if (x < 0 && type == 'u') {
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6686 type = 'd';
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6687 }
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6688 if (x < 0 && (type == 'x' || type == 'X' || type == 'o'))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6689 sign = "-";
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6690 else
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6691 sign = "";
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6692 if (prec < 0)
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6693 prec = 1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6694
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6695 /* buf = '+'/'-'/'' + '0'/'0x'/'' + '[0-9]'*max(prec, len(x in octal))
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6696 * worst case buf = '-0x' + [0-9]*prec, where prec >= 11
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6697 */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6698 if (buflen <= 14 || buflen <= (size_t)3 + (size_t)prec) {
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6699 PyErr_SetString(PyExc_OverflowError,
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6700 "formatted integer is too long (precision too large?)");
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6701 return -1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6702 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6703
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6704 if ((flags & F_ALT) &&
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6705 (type == 'x' || type == 'X')) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6706 /* When converting under %#x or %#X, there are a number
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6707 * of issues that cause pain:
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6708 * - when 0 is being converted, the C standard leaves off
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6709 * the '0x' or '0X', which is inconsistent with other
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6710 * %#x/%#X conversions and inconsistent with Python's
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6711 * hex() function
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6712 * - there are platforms that violate the standard and
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6713 * convert 0 with the '0x' or '0X'
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6714 * (Metrowerks, Compaq Tru64)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6715 * - there are platforms that give '0x' when converting
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6716 * under %#X, but convert 0 in accordance with the
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6717 * standard (OS/2 EMX)
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6718 *
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6719 * We can achieve the desired consistency by inserting our
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6720 * own '0x' or '0X' prefix, and substituting %x/%X in place
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6721 * of %#x/%#X.
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6722 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6723 * Note that this is the same approach as used in
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6724 * formatint() in stringobject.c
22053
49197fd76362 OS/2 EMX port changes (Objects part of patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 21886
diff changeset
6725 */
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6726 PyOS_snprintf(fmt, sizeof(fmt), "%s0%c%%.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6727 sign, type, prec, type);
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6728 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6729 else {
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6730 PyOS_snprintf(fmt, sizeof(fmt), "%s%%%s.%dl%c",
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6731 sign, (flags&F_ALT) ? "#" : "",
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6732 prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6733 }
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6734 if (sign[0])
36262
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6735 return longtounicode(buf, buflen, fmt, -x);
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
6736 else
36262
76a1c20e830e Backport:
Neal Norwitz <nnorwitz@gmail.com>
parents: 36055
diff changeset
6737 return longtounicode(buf, buflen, fmt, x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6738 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6739
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6740 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6741 formatchar(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6742 size_t buflen,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6743 PyObject *v)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6744 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6745 /* presume that the buffer is at least 2 characters long */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6746 if (PyUnicode_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6747 if (PyUnicode_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6748 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6749 buf[0] = PyUnicode_AS_UNICODE(v)[0];
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6750 }
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6751
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6752 else if (PyString_Check(v)) {
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6753 if (PyString_GET_SIZE(v) != 1)
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6754 goto onError;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6755 buf[0] = (Py_UNICODE)PyString_AS_STRING(v)[0];
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6756 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6757
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6758 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6759 /* Integer input truncated to a character */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6760 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6761 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6762 if (x == -1 && PyErr_Occurred())
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6763 goto onError;
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6764 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6765 if (x < 0 || x > 0x10ffff) {
28494
32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises
Walter Dörwald <walter@livinglogic.de>
parents: 28242
diff changeset
6766 PyErr_SetString(PyExc_OverflowError,
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6767 "%c arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6768 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6769 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6770 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6771 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6772 if (x < 0 || x > 0xffff) {
28494
32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises
Walter Dörwald <walter@livinglogic.de>
parents: 28242
diff changeset
6773 PyErr_SetString(PyExc_OverflowError,
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6774 "%c arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6775 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6776 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6777 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6778 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6779 buf[0] = (Py_UNICODE) x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6780 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6781 buf[1] = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6782 return 1;
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6783
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6784 onError:
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6785 PyErr_SetString(PyExc_TypeError,
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6786 "%c requires int or char");
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6787 return -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6788 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6789
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6790 /* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6791
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6792 FORMATBUFLEN is the length of the buffer in which the floats, ints, &
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6793 chars are formatted. XXX This is a magic number. Each formatting
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6794 routine does bounds checking to ensure no overflow, but a better
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6795 solution may be to malloc a buffer of appropriate size for each
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6796 format. For now, the current solution is sufficient.
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6797 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6798 #define FORMATBUFLEN (size_t)120
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6799
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6800 PyObject *PyUnicode_Format(PyObject *format,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6801 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6802 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6803 Py_UNICODE *fmt, *res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6804 int fmtcnt, rescnt, reslen, arglen, argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6805 int args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6806 PyUnicodeObject *result = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6807 PyObject *dict = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6808 PyObject *uformat;
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6809
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6810 if (format == NULL || args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6811 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6812 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6813 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6814 uformat = PyUnicode_FromObject(format);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6815 if (uformat == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6816 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6817 fmt = PyUnicode_AS_UNICODE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6818 fmtcnt = PyUnicode_GET_SIZE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6819
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6820 reslen = rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6821 result = _PyUnicode_New(reslen);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6822 if (result == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6823 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6824 res = PyUnicode_AS_UNICODE(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6825
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6826 if (PyTuple_Check(args)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6827 arglen = PyTuple_Size(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6828 argidx = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6829 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6830 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6831 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6832 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6833 }
26164
63a08639dae2 Fix SF # 635969, No error "not all arguments converted"
Neal Norwitz <nnorwitz@gmail.com>
parents: 25895
diff changeset
6834 if (args->ob_type->tp_as_mapping && !PyTuple_Check(args) &&
63a08639dae2 Fix SF # 635969, No error "not all arguments converted"
Neal Norwitz <nnorwitz@gmail.com>
parents: 25895
diff changeset
6835 !PyObject_TypeCheck(args, &PyBaseString_Type))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6836 dict = args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6837
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6838 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6839 if (*fmt != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6840 if (--rescnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6841 rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6842 reslen += rescnt;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6843 if (_PyUnicode_Resize(&result, reslen) < 0)
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
6844 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6845 res = PyUnicode_AS_UNICODE(result) + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6846 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6847 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6848 *res++ = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6849 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6850 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6851 /* Got a format specifier */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6852 int flags = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6853 int width = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6854 int prec = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6855 Py_UNICODE c = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6856 Py_UNICODE fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6857 PyObject *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6858 PyObject *temp = NULL;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6859 Py_UNICODE *pbuf;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6860 Py_UNICODE sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6861 int len;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6862 Py_UNICODE formatbuf[FORMATBUFLEN]; /* For format{float,int,char}() */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6863
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6864 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6865 if (*fmt == '(') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6866 Py_UNICODE *keystart;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6867 int keylen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6868 PyObject *key;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6869 int pcount = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6870
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6871 if (dict == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6872 PyErr_SetString(PyExc_TypeError,
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
6873 "format requires a mapping");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6874 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6875 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6876 ++fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6877 --fmtcnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6878 keystart = fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6879 /* Skip over balanced parentheses */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6880 while (pcount > 0 && --fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6881 if (*fmt == ')')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6882 --pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6883 else if (*fmt == '(')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6884 ++pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6885 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6886 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6887 keylen = fmt - keystart - 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6888 if (fmtcnt < 0 || pcount > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6889 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6890 "incomplete format key");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6891 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6892 }
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6893 #if 0
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6894 /* keys are converted to strings using UTF-8 and
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6895 then looked up since Python uses strings to hold
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6896 variables names etc. in its namespaces and we
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6897 wouldn't want to break common idioms. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6898 key = PyUnicode_EncodeUTF8(keystart,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6899 keylen,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6900 NULL);
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6901 #else
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6902 key = PyUnicode_FromUnicode(keystart, keylen);
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6903 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6904 if (key == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6905 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6906 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6907 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6908 args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6909 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6910 args = PyObject_GetItem(dict, key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6911 Py_DECREF(key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6912 if (args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6913 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6914 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6915 args_owned = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6916 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6917 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6918 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6919 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6920 switch (c = *fmt++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6921 case '-': flags |= F_LJUST; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6922 case '+': flags |= F_SIGN; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6923 case ' ': flags |= F_BLANK; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6924 case '#': flags |= F_ALT; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6925 case '0': flags |= F_ZERO; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6926 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6927 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6928 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6929 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6930 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6931 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6932 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6933 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6934 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6935 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6936 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6937 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6938 width = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6939 if (width < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6940 flags |= F_LJUST;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6941 width = -width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6942 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6943 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6944 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6945 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6946 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6947 width = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6948 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6949 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6950 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6951 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6952 if ((width*10) / 10 != width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6953 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6954 "width too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6955 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6956 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6957 width = width*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6958 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6959 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6960 if (c == '.') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6961 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6962 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6963 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6964 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6965 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6966 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6967 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6968 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6969 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6970 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6971 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6972 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6973 prec = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6974 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6975 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6976 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6977 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6978 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6979 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6980 prec = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6981 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6982 c = Py_CHARMASK(*fmt++);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6983 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6984 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6985 if ((prec*10) / 10 != prec) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6986 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6987 "prec too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6988 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6989 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6990 prec = prec*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6991 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6992 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6993 } /* prec */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6994 if (fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6995 if (c == 'h' || c == 'l' || c == 'L') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6996 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6997 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6998 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6999 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7000 if (fmtcnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7001 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7002 "incomplete format");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7003 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7004 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7005 if (c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7006 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7007 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7008 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7009 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7010 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7011 fill = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7012 switch (c) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7013
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7014 case '%':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7015 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7016 /* presume that buffer length is at least 1 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7017 pbuf[0] = '%';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7018 len = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7019 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7020
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7021 case 's':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7022 case 'r':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7023 if (PyUnicode_Check(v) && c == 's') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7024 temp = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7025 Py_INCREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7026 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7027 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7028 PyObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7029 if (c == 's')
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7030 temp = PyObject_Unicode(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7031 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7032 temp = PyObject_Repr(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7033 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7034 goto onError;
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7035 if (PyUnicode_Check(temp))
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7036 /* nothing to do */;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7037 else if (PyString_Check(temp)) {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7038 /* convert to string to Unicode */
37261
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7039 unicode = PyUnicode_Decode(PyString_AS_STRING(temp),
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7040 PyString_GET_SIZE(temp),
37261
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7041 NULL,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7042 "strict");
37261
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7043 Py_DECREF(temp);
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7044 temp = unicode;
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7045 if (temp == NULL)
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7046 goto onError;
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7047 }
32801
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7048 else {
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7049 Py_DECREF(temp);
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7050 PyErr_SetString(PyExc_TypeError,
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7051 "%s argument has non-string str()");
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7052 goto onError;
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7053 }
16de4f90ef44 Let u'%s' % obj try obj.__unicode__() first and fallback to obj.__str__().
Marc-André Lemburg <mal@egenix.com>
parents: 32695
diff changeset
7054 }
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7055 pbuf = PyUnicode_AS_UNICODE(temp);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7056 len = PyUnicode_GET_SIZE(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7057 if (prec >= 0 && len > prec)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7058 len = prec;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7059 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7060
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7061 case 'i':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7062 case 'd':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7063 case 'u':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7064 case 'o':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7065 case 'x':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7066 case 'X':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7067 if (c == 'i')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7068 c = 'd';
15455
d0146d128897 Fox for SF bug #123859: %[duxXo] long formats inconsistent.
Tim Peters <tim.peters@gmail.com>
parents: 15001
diff changeset
7069 if (PyLong_Check(v)) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7070 temp = formatlong(v, flags, prec, c);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7071 if (!temp)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7072 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7073 pbuf = PyUnicode_AS_UNICODE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7074 len = PyUnicode_GET_SIZE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7075 sign = 1;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7076 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7077 else {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7078 pbuf = formatbuf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7079 len = formatint(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7080 flags, prec, c, v);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7081 if (len < 0)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7082 goto onError;
31042
9c846c3df6f3 - Removed FutureWarnings related to hex/oct literals and conversions
Guido van Rossum <guido@python.org>
parents: 31021
diff changeset
7083 sign = 1;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7084 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7085 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7086 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7087 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7088
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7089 case 'e':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7090 case 'E':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7091 case 'f':
30216
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
7092 case 'F':
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7093 case 'g':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7094 case 'G':
30216
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
7095 if (c == 'F')
7ab1a41f16da SF bug #795506: Wrong handling of string format code for float values.
Raymond Hettinger <python@rcn.com>
parents: 30164
diff changeset
7096 c = 'f';
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7097 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7098 len = formatfloat(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7099 flags, prec, c, v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7100 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7101 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7102 sign = 1;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7103 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7104 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7105 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7106
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7107 case 'c':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7108 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7109 len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7110 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7111 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7112 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7113
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7114 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7115 PyErr_Format(PyExc_ValueError,
15579
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the
Andrew M. Kuchling <amk@amk.ca>
parents: 15455
diff changeset
7116 "unsupported format character '%c' (0x%x) "
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the
Andrew M. Kuchling <amk@amk.ca>
parents: 15455
diff changeset
7117 "at index %i",
30441
76e998aee644 On c.l.py, Martin v. Löwis said that Py_UNICODE could be of a signed type,
Tim Peters <tim.peters@gmail.com>
parents: 30439
diff changeset
7118 (31<=c && c<=126) ? (char)c : '?',
25375
4df0e691f4da Add cast to avoid compiler warning.
Marc-André Lemburg <mal@egenix.com>
parents: 25248
diff changeset
7119 (int)c,
25233
8d751b62ecc4 Fix warnings on 64-bit platforms about casts from pointers to ints.
Guido van Rossum <guido@python.org>
parents: 25120
diff changeset
7120 (int)(fmt -1 - PyUnicode_AS_UNICODE(uformat)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7121 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7122 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7123 if (sign) {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7124 if (*pbuf == '-' || *pbuf == '+') {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
7125 sign = *pbuf++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7126 len--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7127 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7128 else if (flags & F_SIGN)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7129 sign = '+';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7130 else if (flags & F_BLANK)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7131 sign = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7132 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7133 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7134 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7135 if (width < len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7136 width = len;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
7137 if (rescnt - (sign != 0) < width) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7138 reslen -= rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7139 rescnt = width + fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7140 reslen += rescnt;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
7141 if (reslen < 0) {
36878
accfb156f9cc Backport r42894: SF #1444030 Fix several potential defects found
Hye-Shik Chang <hyeshik@gmail.com>
parents: 36262
diff changeset
7142 Py_XDECREF(temp);
37261
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7143 PyErr_NoMemory();
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7144 goto onError;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
7145 }
37261
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7146 if (_PyUnicode_Resize(&result, reslen) < 0) {
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7147 Py_XDECREF(temp);
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7148 goto onError;
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7149 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7150 res = PyUnicode_AS_UNICODE(result)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7151 + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7152 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7153 if (sign) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7154 if (fill != ' ')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7155 *res++ = sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7156 rescnt--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7157 if (width > len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7158 width--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7159 }
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7160 if ((flags & F_ALT) && (c == 'x' || c == 'X')) {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7161 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7162 assert(pbuf[1] == c);
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7163 if (fill != ' ') {
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7164 *res++ = *pbuf++;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7165 *res++ = *pbuf++;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7166 }
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7167 rescnt -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7168 width -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7169 if (width < 0)
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7170 width = 0;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7171 len -= 2;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7172 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7173 if (width > len && !(flags & F_LJUST)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7174 do {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7175 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7176 *res++ = fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7177 } while (--width > len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7178 }
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7179 if (fill == ' ') {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7180 if (sign)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7181 *res++ = sign;
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7182 if ((flags & F_ALT) && (c == 'x' || c == 'X')) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7183 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
7184 assert(pbuf[1] == c);
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7185 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7186 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7187 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
7188 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7189 Py_UNICODE_COPY(res, pbuf, len);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7190 res += len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7191 rescnt -= len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7192 while (--width >= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7193 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7194 *res++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7195 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7196 if (dict && (argidx < arglen) && c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7197 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
7198 "not all arguments converted during string formatting");
37261
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7199 Py_XDECREF(temp);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7200 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7202 Py_XDECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7203 } /* '%' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7204 } /* until end */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7205 if (argidx < arglen && !dict) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7206 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
7207 "not all arguments converted during string formatting");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7208 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7209 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7210
37261
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7211 if (_PyUnicode_Resize(&result, reslen - rescnt) < 0)
bc5161f292b7 Fix the refleak from test_unicode.
Neal Norwitz <nnorwitz@gmail.com>
parents: 36878
diff changeset
7212 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7213 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7214 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7215 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7216 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7217 return (PyObject *)result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7218
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7219 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7220 Py_XDECREF(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7221 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7222 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7223 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7224 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7225 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7226 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7227
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7228 static PyBufferProcs unicode_as_buffer = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7229 (getreadbufferproc) unicode_buffer_getreadbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7230 (getwritebufferproc) unicode_buffer_getwritebuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7231 (getsegcountproc) unicode_buffer_getsegcount,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7232 (getcharbufferproc) unicode_buffer_getcharbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7233 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7234
24280
1598a1e473e2 staticforward bites the dust.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24071
diff changeset
7235 static PyObject *
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7236 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7237
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7238 static PyObject *
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7239 unicode_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7240 {
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7241 PyObject *x = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7242 static char *kwlist[] = {"string", "encoding", "errors", 0};
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7243 char *encoding = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7244 char *errors = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7245
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7246 if (type != &PyUnicode_Type)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7247 return unicode_subtype_new(type, args, kwds);
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7248 if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:unicode",
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7249 kwlist, &x, &encoding, &errors))
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7250 return NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7251 if (x == NULL)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7252 return (PyObject *)_PyUnicode_New(0);
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
7253 if (encoding == NULL && errors == NULL)
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
7254 return PyObject_Unicode(x);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
7255 else
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7256 return PyUnicode_FromEncodedObject(x, encoding, errors);
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7257 }
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7258
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7259 static PyObject *
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7260 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7261 {
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7262 PyUnicodeObject *tmp, *pnew;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7263 int n;
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7264
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7265 assert(PyType_IsSubtype(type, &PyUnicode_Type));
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7266 tmp = (PyUnicodeObject *)unicode_new(&PyUnicode_Type, args, kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7267 if (tmp == NULL)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7268 return NULL;
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7269 assert(PyUnicode_Check(tmp));
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7270 pnew = (PyUnicodeObject *) type->tp_alloc(type, n = tmp->length);
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
7271 if (pnew == NULL) {
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
7272 Py_DECREF(tmp);
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7273 return NULL;
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
7274 }
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7275 pnew->str = PyMem_NEW(Py_UNICODE, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7276 if (pnew->str == NULL) {
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7277 _Py_ForgetReference((PyObject *)pnew);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
7278 PyObject_Del(pnew);
29641
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new
Raymond Hettinger <python@rcn.com>
parents: 29164
diff changeset
7279 Py_DECREF(tmp);
27826
7dffcc498c45 Add more missing PyErr_NoMemory() after failled memory allocs
Neal Norwitz <nnorwitz@gmail.com>
parents: 27774
diff changeset
7280 return PyErr_NoMemory();
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7281 }
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7282 Py_UNICODE_COPY(pnew->str, tmp->str, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7283 pnew->length = n;
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7284 pnew->hash = tmp->hash;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7285 Py_DECREF(tmp);
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
7286 return (PyObject *)pnew;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7287 }
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
7288
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
7289 PyDoc_STRVAR(unicode_doc,
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7290 "unicode(string [, encoding[, errors]]) -> object\n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7291 \n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7292 Create a new Unicode object from the given encoded string.\n\
24392
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
7293 encoding defaults to the current default string encoding.\n\
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
7294 errors can be 'strict', 'replace' or 'ignore' and defaults to 'strict'.");
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7295
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7296 PyTypeObject PyUnicode_Type = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7297 PyObject_HEAD_INIT(&PyType_Type)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7298 0, /* ob_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7299 "unicode", /* tp_name */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7300 sizeof(PyUnicodeObject), /* tp_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7301 0, /* tp_itemsize */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7302 /* Slots */
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
7303 (destructor)unicode_dealloc, /* tp_dealloc */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7304 0, /* tp_print */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7305 0, /* tp_getattr */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7306 0, /* tp_setattr */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7307 (cmpfunc) unicode_compare, /* tp_compare */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7308 (reprfunc) unicode_repr, /* tp_repr */
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7309 &unicode_as_number, /* tp_as_number */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7310 &unicode_as_sequence, /* tp_as_sequence */
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
7311 &unicode_as_mapping, /* tp_as_mapping */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7312 (hashfunc) unicode_hash, /* tp_hash*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7313 0, /* tp_call*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7314 (reprfunc) unicode_str, /* tp_str */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7315 PyObject_GenericGetAttr, /* tp_getattro */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7316 0, /* tp_setattro */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7317 &unicode_as_buffer, /* tp_as_buffer */
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7318 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES |
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
7319 Py_TPFLAGS_BASETYPE, /* tp_flags */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7320 unicode_doc, /* tp_doc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7321 0, /* tp_traverse */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7322 0, /* tp_clear */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7323 0, /* tp_richcompare */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7324 0, /* tp_weaklistoffset */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7325 0, /* tp_iter */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7326 0, /* tp_iternext */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7327 unicode_methods, /* tp_methods */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7328 0, /* tp_members */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7329 0, /* tp_getset */
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
7330 &PyBaseString_Type, /* tp_base */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7331 0, /* tp_dict */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7332 0, /* tp_descr_get */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7333 0, /* tp_descr_set */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7334 0, /* tp_dictoffset */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7335 0, /* tp_init */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7336 0, /* tp_alloc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
7337 unicode_new, /* tp_new */
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
7338 PyObject_Del, /* tp_free */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7339 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7340
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7341 /* Initialize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7342
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
7343 void _PyUnicode_Init(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7344 {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7345 int i;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7346
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
7347 /* Init the implementation */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
7348 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
7349 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7350 unicode_empty = _PyUnicode_New(0);
12876
243593f11fcf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12646
diff changeset
7351 strcpy(unicode_default_encoding, "ascii");
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7352 for (i = 0; i < 256; i++)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7353 unicode_latin1[i] = NULL;
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
7354 if (PyType_Ready(&PyUnicode_Type) < 0)
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
7355 Py_FatalError("Can't initialize 'unicode'");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7356 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7357
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7358 /* Finalize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7359
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7360 void
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
7361 _PyUnicode_Fini(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7362 {
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
7363 PyUnicodeObject *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7364 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7365
14998
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
7366 Py_XDECREF(unicode_empty);
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
7367 unicode_empty = NULL;
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
7368
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7369 for (i = 0; i < 256; i++) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7370 if (unicode_latin1[i]) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7371 Py_DECREF(unicode_latin1[i]);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7372 unicode_latin1[i] = NULL;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7373 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7374 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
7375
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
7376 for (u = unicode_freelist; u != NULL;) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7377 PyUnicodeObject *v = u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7378 u = *(PyUnicodeObject **)u;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
7379 if (v->str)
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
7380 PyMem_DEL(v->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
7381 Py_XDECREF(v->defenc);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
7382 PyObject_Del(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7383 }
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
7384 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
7385 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
7386 }
29164
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
7387
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
7388 /*
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
7389 Local variables:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
7390 c-basic-offset: 4
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
7391 indent-tabs-mode: nil
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
7392 End:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514.
Martin v. Löwis <martin@v.loewis.de>
parents: 28575
diff changeset
7393 */