Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodeobject.c @ 27022:e2c88c451c19 legacy-trunk

Find changesets by keywords (author, files, the commit message), revision number or hash, or revset expression.
Replaced imp.set_frozenmodules() cruft with proper zipimport support. This work uncovered the zipimport bug in 2.3a1 -- wish I'd had time to do this before the release :-(.
author Just van Rossum <just@letterror.com>
date Thu, 02 Jan 2003 13:13:01 +0000
parents e153961f961f
children adf4ab3eaa62
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
rev   line source
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1 /*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3 Unicode implementation based on original code by Fredrik Lundh,
12646
17d1a3f1f085 Minimal change so I can add the rest of MAL's checkin message:
Fred Drake <fdrake@acm.org>
parents: 12645
diff changeset
4 modified by Marc-Andre Lemburg <mal@lemburg.com> according to the
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5 Unicode Integration Proposal (see file Misc/unicode.txt).
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6
13828
254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions.
Guido van Rossum <guido@python.org>
parents: 13761
diff changeset
7 Copyright (c) Corporation for National Research Initiatives.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
8
16029
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
9 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
10 The original string type implementation is:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
11
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
12 Copyright (c) 1999 by Secret Labs AB
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
13 Copyright (c) 1999 by Fredrik Lundh
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
14
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
15 By obtaining, using, and/or copying this software and/or its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
16 associated documentation, you agree that you have read, understood,
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
17 and will comply with the following terms and conditions:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
18
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
19 Permission to use, copy, modify, and distribute this software and its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
20 associated documentation for any purpose and without fee is hereby
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
21 granted, provided that the above copyright notice appears in all
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
22 copies, and that both that copyright notice and this permission notice
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
23 appear in supporting documentation, and that the name of Secret Labs
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
24 AB or the author not be used in advertising or publicity pertaining to
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
25 distribution of the software without specific, written prior
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
26 permission.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
27
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
28 SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
29 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
30 FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
31 ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
32 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
33 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
34 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
35 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
36
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the
Fredrik Lundh <fredrik@pythonware.com>
parents: 15952
diff changeset
37 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
38
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
39 #include "Python.h"
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
40
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
41 #include "unicodeobject.h"
13209
b11e0a898de0 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13196
diff changeset
42 #include "ucnhash.h"
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
43
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
44 #ifdef MS_WINDOWS
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
45 #include <windows.h>
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
46 #endif
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
47
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
48 /* Limit for the Unicode object free list */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
49
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
50 #define MAX_UNICODE_FREELIST_SIZE 1024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
51
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
52 /* Limit for the Unicode object free list stay alive optimization.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
53
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
54 The implementation will keep allocated Unicode memory intact for
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
55 all objects on the free list having a size less than this
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
56 limit. This reduces malloc() overhead for small Unicode objects.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
57
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
58 At worst this will result in MAX_UNICODE_FREELIST_SIZE *
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
59 (sizeof(PyUnicodeObject) + KEEPALIVE_SIZE_LIMIT +
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
60 malloc()-overhead) bytes of unused garbage.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
61
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
62 Setting the limit to 0 effectively turns the feature off.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
63
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
64 Note: This is an experimental feature ! If you get core dumps when
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
65 using Unicode objects, turn this feature off.
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
66
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
67 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
68
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
69 #define KEEPALIVE_SIZE_LIMIT 9
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
70
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
71 /* Endianness switches; defaults to little endian */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
72
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
73 #ifdef WORDS_BIGENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
74 # define BYTEORDER_IS_BIG_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
75 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
76 # define BYTEORDER_IS_LITTLE_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
77 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
78
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
79 /* --- Globals ------------------------------------------------------------
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
80
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
81 The globals are initialized by the _PyUnicode_Init() API and should
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
82 not be used before calling that API.
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
83
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
84 */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
85
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
86 /* Free list for Unicode objects */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
87 static PyUnicodeObject *unicode_freelist;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
88 static int unicode_freelist_size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
89
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
90 /* The empty Unicode object is shared to improve performance. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
91 static PyUnicodeObject *unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
92
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
93 /* Single character Unicode strings in the Latin-1 range are being
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
94 shared as well. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
95 static PyUnicodeObject *unicode_latin1[256];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
96
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
97 /* Default encoding to use and assume when NULL is passed as encoding
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
98 parameter; it is initialized by _PyUnicode_Init().
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
99
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
100 Always use the PyUnicode_SetDefaultEncoding() and
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
101 PyUnicode_GetDefaultEncoding() APIs to access this global.
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
102
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
103 */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
104 static char unicode_default_encoding[100];
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
105
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
106 Py_UNICODE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
107 PyUnicode_GetMax(void)
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
108 {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
109 #ifdef Py_UNICODE_WIDE
18298
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
110 return 0x10FFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
111 #else
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
112 /* This is actually an illegal character, so it should
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
113 not be passed to unichr. */
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
114 return 0xFFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
115 #endif
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
116 }
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE.
Martin v. Löwis <martin@v.loewis.de>
parents: 18295
diff changeset
117
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
118 /* --- Unicode Object ----------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
119
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
120 static
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
121 int unicode_resize(register PyUnicodeObject *unicode,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
122 int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
123 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
124 void *oldstr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
125
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
126 /* Shortcut if there's nothing much to do. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
127 if (unicode->length == length)
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
128 goto reset;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
129
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
130 /* Resizing shared object (unicode_empty or single character
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
131 objects) in-place is not allowed. Use PyUnicode_Resize()
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
132 instead ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
133 if (unicode == unicode_empty ||
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
134 (unicode->length == 1 &&
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
135 unicode->str[0] < 256 &&
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
136 unicode_latin1[unicode->str[0]] == unicode)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
137 PyErr_SetString(PyExc_SystemError,
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
138 "can't resize shared unicode objects");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
139 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
140 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
141
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
142 /* We allocate one more byte to make sure the string is
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
143 Ux0000 terminated -- XXX is this needed ? */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
144 oldstr = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
145 PyMem_RESIZE(unicode->str, Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
146 if (!unicode->str) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
147 unicode->str = oldstr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
148 PyErr_NoMemory();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
149 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
150 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
151 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
152 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
153
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
154 reset:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
155 /* Reset the object caches */
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
156 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
157 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
158 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
159 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
160 unicode->hash = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
161
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
162 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
163 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
164
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
165 /* We allocate one more byte to make sure the string is
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
166 Ux0000 terminated -- XXX is this needed ?
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
167
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
168 XXX This allocator could further be enhanced by assuring that the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
169 free list never reduces its size below 1.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
170
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
171 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
172
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
173 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
174 PyUnicodeObject *_PyUnicode_New(int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
175 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
176 register PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
177
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
178 /* Optimization for empty strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
179 if (length == 0 && unicode_empty != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
180 Py_INCREF(unicode_empty);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
181 return unicode_empty;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
182 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
183
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
184 /* Unicode freelist & memory allocation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
185 if (unicode_freelist) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
186 unicode = unicode_freelist;
12939
88004f25dc25 Vladimir MARANGOZOV <Vladimir.Marangozov@inrialpes.fr>:
Marc-André Lemburg <mal@egenix.com>
parents: 12924
diff changeset
187 unicode_freelist = *(PyUnicodeObject **)unicode;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
188 unicode_freelist_size--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
189 if (unicode->str) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
190 /* Keep-Alive optimization: we only upsize the buffer,
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
191 never downsize it. */
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
192 if ((unicode->length < length) &&
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
193 unicode_resize(unicode, length)) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
194 PyMem_DEL(unicode->str);
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
195 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
196 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
197 }
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
198 else {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
199 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
18120
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
200 }
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at
Guido van Rossum <guido@python.org>
parents: 18051
diff changeset
201 PyObject_INIT(unicode, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
202 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
203 else {
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
204 unicode = PyObject_New(PyUnicodeObject, &PyUnicode_Type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
205 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
206 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
207 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
208 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
209
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
210 if (!unicode->str) {
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
211 PyErr_NoMemory();
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
212 goto onError;
12511
8747f1c8b5fb Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12369
diff changeset
213 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
214 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
215 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
216 unicode->hash = -1;
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
217 unicode->defenc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
218 return unicode;
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
219
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
220 onError:
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
221 _Py_ForgetReference((PyObject *)unicode);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
222 PyObject_Del(unicode);
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
223 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
224 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
226 static
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
227 void unicode_dealloc(register PyUnicodeObject *unicode)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
228 {
21262
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
229 if (PyUnicode_CheckExact(unicode) &&
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
230 unicode_freelist_size < MAX_UNICODE_FREELIST_SIZE) {
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
231 /* Keep-Alive optimization */
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
232 if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
233 PyMem_DEL(unicode->str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
234 unicode->str = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
235 unicode->length = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
236 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
237 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
238 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
239 unicode->defenc = NULL;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
240 }
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
241 /* Add to free list */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
242 *(PyUnicodeObject **)unicode = unicode_freelist;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
243 unicode_freelist = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
244 unicode_freelist_size++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
245 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
246 else {
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
247 PyMem_DEL(unicode->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
248 Py_XDECREF(unicode->defenc);
21262
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode).
Guido van Rossum <guido@python.org>
parents: 21082
diff changeset
249 unicode->ob_type->tp_free((PyObject *)unicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
250 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
251 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
252
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
253 int PyUnicode_Resize(PyObject **unicode,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
254 int length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
255 {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
256 register PyUnicodeObject *v;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
257
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
258 /* Argument checks */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
259 if (unicode == NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
260 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
261 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
262 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
263 v = (PyUnicodeObject *)*unicode;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
264 if (v == NULL || !PyUnicode_Check(v) || v->ob_refcnt != 1 || length < 0) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
265 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
266 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
267 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
268
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
269 /* Resizing unicode_empty and single character objects is not
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
270 possible since these are being shared. We simply return a fresh
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
271 copy with the same Unicode content. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
272 if (v->length != length &&
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
273 (v == unicode_empty || v->length == 1)) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
274 PyUnicodeObject *w = _PyUnicode_New(length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
275 if (w == NULL)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
276 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
277 Py_UNICODE_COPY(w->str, v->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
278 length < v->length ? length : v->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
279 *unicode = (PyObject *)w;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
280 return 0;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
281 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
282
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
283 /* Note that we don't have to modify *unicode for unshared Unicode
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
284 objects, since we can modify them in-place. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
285 return unicode_resize(v, length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
286 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
287
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
288 /* Internal API for use in unicodeobject.c only ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
289 #define _PyUnicode_Resize(unicodevar, length) \
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
290 PyUnicode_Resize(((PyObject **)(unicodevar)), length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
291
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
292 PyObject *PyUnicode_FromUnicode(const Py_UNICODE *u,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
293 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
294 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
295 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
296
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
297 /* If the Unicode data is known at construction time, we can apply
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
298 some optimizations which share commonly used objects. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
299 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
300
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
301 /* Optimization for empty strings */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
302 if (size == 0 && unicode_empty != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
303 Py_INCREF(unicode_empty);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
304 return (PyObject *)unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
305 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
306
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
307 /* Single character Unicode objects in the Latin-1 range are
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
308 shared when using this constructor */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
309 if (size == 1 && *u < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
310 unicode = unicode_latin1[*u];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
311 if (!unicode) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
312 unicode = _PyUnicode_New(1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
313 if (!unicode)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
314 return NULL;
18051
e897550b4748 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode.
Marc-André Lemburg <mal@egenix.com>
parents: 17995
diff changeset
315 unicode->str[0] = *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
316 unicode_latin1[*u] = unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
317 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
318 Py_INCREF(unicode);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
319 return (PyObject *)unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
320 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
321 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
322
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
323 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
324 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
325 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
326
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
327 /* Copy the Unicode data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
328 if (u != NULL)
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
329 Py_UNICODE_COPY(unicode->str, u, size);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
330
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
331 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
332 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
333
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
334 #ifdef HAVE_WCHAR_H
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
335
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
336 PyObject *PyUnicode_FromWideChar(register const wchar_t *w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
337 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
338 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
339 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
340
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
341 if (w == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
342 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
343 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
344 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
345
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
346 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
347 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
348 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
349
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
350 /* Copy the wchar_t data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
351 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
352 memcpy(unicode->str, w, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
353 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
354 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
355 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
356 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
357 u = PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
358 for (i = size; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
359 *u++ = *w++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
360 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
361 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
362
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
363 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
364 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
365
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
366 int PyUnicode_AsWideChar(PyUnicodeObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
367 register wchar_t *w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
368 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
369 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
370 if (unicode == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
371 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
372 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
373 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
374 if (size > PyUnicode_GET_SIZE(unicode))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
375 size = PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
376 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
377 memcpy(w, unicode->str, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
378 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
379 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
380 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
381 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
382 u = PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
383 for (i = size; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
384 *w++ = *u++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
385 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
386 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
387
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
388 return size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
389 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
390
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
391 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
392
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
393 PyObject *PyUnicode_FromOrdinal(int ordinal)
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
394 {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
395 Py_UNICODE s[2];
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
396
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
397 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
398 if (ordinal < 0 || ordinal > 0x10ffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
399 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
400 "unichr() arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
401 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
402 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
403 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
404 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
405 if (ordinal < 0 || ordinal > 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
406 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
407 "unichr() arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
408 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
409 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
410 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
411 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
412
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
413 if (ordinal <= 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
414 /* UCS-2 character */
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
415 s[0] = (Py_UNICODE) ordinal;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
416 return PyUnicode_FromUnicode(s, 1);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
417 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
418 else {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
419 #ifndef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
420 /* UCS-4 character. store as two surrogate characters */
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
421 ordinal -= 0x10000L;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
422 s[0] = 0xD800 + (Py_UNICODE) (ordinal >> 10);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
423 s[1] = 0xDC00 + (Py_UNICODE) (ordinal & 0x03FF);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
424 return PyUnicode_FromUnicode(s, 2);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
425 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
426 s[0] = (Py_UNICODE)ordinal;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
427 return PyUnicode_FromUnicode(s, 1);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
428 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
429 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
430 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
431
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
432 PyObject *PyUnicode_FromObject(register PyObject *obj)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
433 {
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
434 /* XXX Perhaps we should make this API an alias of
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
435 PyObject_Unicode() instead ?! */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
436 if (PyUnicode_CheckExact(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
437 Py_INCREF(obj);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
438 return obj;
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
439 }
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
440 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
441 /* For a Unicode subtype that's not a Unicode object,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
442 return a true Unicode object with the same data. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
443 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(obj),
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
444 PyUnicode_GET_SIZE(obj));
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
445 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
446 return PyUnicode_FromEncodedObject(obj, NULL, "strict");
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
447 }
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
448
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
449 PyObject *PyUnicode_FromEncodedObject(register PyObject *obj,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
450 const char *encoding,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
451 const char *errors)
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
452 {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
453 const char *s = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
454 int len;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
455 int owned = 0;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
456 PyObject *v;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
457
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
458 if (obj == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
459 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
460 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
461 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
462
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
463 #if 0
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
464 /* For b/w compatibility we also accept Unicode objects provided
20563
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
465 that no encodings is given and then redirect to
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
466 PyObject_Unicode() which then applies the additional logic for
9058e49c6c57 Additional test and documentation for the unicode() changes.
Marc-André Lemburg <mal@egenix.com>
parents: 20554
diff changeset
467 Unicode subclasses.
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
468
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
469 NOTE: This API should really only be used for object which
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
470 represent *encoded* Unicode !
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
471
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
472 */
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
473 if (PyUnicode_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
474 if (encoding) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
475 PyErr_SetString(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
476 "decoding Unicode is not supported");
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
477 return NULL;
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
478 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
479 return PyObject_Unicode(obj);
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
480 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
481 #else
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
482 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
483 PyErr_SetString(PyExc_TypeError,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
484 "decoding Unicode is not supported");
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
485 return NULL;
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
486 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
487 #endif
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
488
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
489 /* Coerce object */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
490 if (PyString_Check(obj)) {
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
491 s = PyString_AS_STRING(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
492 len = PyString_GET_SIZE(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
493 }
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
494 else if (PyObject_AsCharBuffer(obj, &s, &len)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
495 /* Overwrite the error message with something more useful in
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
496 case of a TypeError. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
497 if (PyErr_ExceptionMatches(PyExc_TypeError))
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
498 PyErr_Format(PyExc_TypeError,
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
499 "coercing to Unicode: need string or buffer, "
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
500 "%.80s found",
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
501 obj->ob_type->tp_name);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
502 goto onError;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
503 }
19999
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now
Marc-André Lemburg <mal@egenix.com>
parents: 19998
diff changeset
504
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
505 /* Convert to Unicode */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
506 if (len == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
507 Py_INCREF(unicode_empty);
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
508 v = (PyObject *)unicode_empty;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
509 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
510 else
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
511 v = PyUnicode_Decode(s, len, encoding, errors);
15952
1f92d63b20f0 This patch adds a new builtin unistr() which behaves like str()
Marc-André Lemburg <mal@egenix.com>
parents: 15910
diff changeset
512
13633
e1469b7a2e68 gcc is being stupid with if/else constructs
Greg Stein <gstein@lyra.org>
parents: 13628
diff changeset
513 if (owned) {
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
514 Py_DECREF(obj);
13633
e1469b7a2e68 gcc is being stupid with if/else constructs
Greg Stein <gstein@lyra.org>
parents: 13628
diff changeset
515 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
516 return v;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
517
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
518 onError:
13633
e1469b7a2e68 gcc is being stupid with if/else constructs
Greg Stein <gstein@lyra.org>
parents: 13628
diff changeset
519 if (owned) {
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
520 Py_DECREF(obj);
13633
e1469b7a2e68 gcc is being stupid with if/else constructs
Greg Stein <gstein@lyra.org>
parents: 13628
diff changeset
521 }
13401
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding
Marc-André Lemburg <mal@egenix.com>
parents: 13396
diff changeset
522 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
523 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
524
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
525 PyObject *PyUnicode_Decode(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
526 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
527 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
528 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
529 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
530 PyObject *buffer = NULL, *unicode;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
531
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
532 if (encoding == NULL)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
533 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
534
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
535 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
536 if (strcmp(encoding, "utf-8") == 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
537 return PyUnicode_DecodeUTF8(s, size, errors);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
538 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
539 return PyUnicode_DecodeLatin1(s, size, errors);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
540 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
541 return PyUnicode_DecodeASCII(s, size, errors);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
542
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
543 /* Decode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
544 buffer = PyBuffer_FromMemory((void *)s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
545 if (buffer == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
546 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
547 unicode = PyCodec_Decode(buffer, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
548 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
549 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
550 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
551 PyErr_Format(PyExc_TypeError,
12320
5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format
Guido van Rossum <guido@python.org>
parents: 12275
diff changeset
552 "decoder did not return an unicode object (type=%.400s)",
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
553 unicode->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
554 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
555 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
556 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
557 Py_DECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
558 return unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
559
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
560 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
561 Py_XDECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
562 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
563 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
564
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
565 PyObject *PyUnicode_Encode(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
566 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
567 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
568 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
569 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
570 PyObject *v, *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
571
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
572 unicode = PyUnicode_FromUnicode(s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
573 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
574 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
575 v = PyUnicode_AsEncodedString(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
576 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
577 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
578 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
579
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
580 PyObject *PyUnicode_AsEncodedString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
581 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
582 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
583 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
584 PyObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
585
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
586 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
587 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
588 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
589 }
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
590
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
591 if (encoding == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
592 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
593
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
594 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
595 if (errors == NULL) {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
596 if (strcmp(encoding, "utf-8") == 0)
17995
837825a779c1 fix bogus indentation
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 17887
diff changeset
597 return PyUnicode_AsUTF8String(unicode);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
598 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
599 return PyUnicode_AsLatin1String(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
600 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
601 return PyUnicode_AsASCIIString(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
602 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
603
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
604 /* Encode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
605 v = PyCodec_Encode(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
606 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
607 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
608 /* XXX Should we really enforce this ? */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
609 if (!PyString_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
610 PyErr_Format(PyExc_TypeError,
12320
5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format
Guido van Rossum <guido@python.org>
parents: 12275
diff changeset
611 "encoder did not return a string object (type=%.400s)",
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
612 v->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
613 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
614 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
615 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
616 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
617
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
618 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
619 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
620 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
621
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
622 PyObject *_PyUnicode_AsDefaultEncodedString(PyObject *unicode,
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
623 const char *errors)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
624 {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
625 PyObject *v = ((PyUnicodeObject *)unicode)->defenc;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
626
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
627 if (v)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
628 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
629 v = PyUnicode_AsEncodedString(unicode, NULL, errors);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
630 if (v && errors == NULL)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
631 ((PyUnicodeObject *)unicode)->defenc = v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
632 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
633 }
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
634
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
635 Py_UNICODE *PyUnicode_AsUnicode(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
636 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
637 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
638 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
639 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
640 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
641 return PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
642
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
643 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
644 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
645 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
646
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
647 int PyUnicode_GetSize(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
648 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
649 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
650 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
651 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
652 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
653 return PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
654
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
655 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
656 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
657 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
658
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
659 const char *PyUnicode_GetDefaultEncoding(void)
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
660 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
661 return unicode_default_encoding;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
662 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
663
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
664 int PyUnicode_SetDefaultEncoding(const char *encoding)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
665 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
666 PyObject *v;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
667
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
668 /* Make sure the encoding is valid. As side effect, this also
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
669 loads the encoding into the codec registry cache. */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
670 v = _PyCodec_Lookup(encoding);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
671 if (v == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
672 goto onError;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
673 Py_DECREF(v);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
674 strncpy(unicode_default_encoding,
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
675 encoding,
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
676 sizeof(unicode_default_encoding));
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
677 return 0;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
678
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
679 onError:
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
680 return -1;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
681 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
682
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
683 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
684 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
685 if no exception occured, copy the replacement to the output
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
686 and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
687 return 0 on success, -1 on error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
688 */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
689
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
690 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
691 int unicode_decode_call_errorhandler(const char *errors, PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
692 const char *encoding, const char *reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
693 const char *input, int insize, int *startinpos, int *endinpos, PyObject **exceptionObject, const char **inptr,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
694 PyObject **output, int *outpos, Py_UNICODE **outptr)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
695 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
696 static char *argparse = "O!i;decoding error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
697
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
698 PyObject *restuple = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
699 PyObject *repunicode = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
700 int outsize = PyUnicode_GET_SIZE(*output);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
701 int requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
702 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
703 Py_UNICODE *repptr;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
704 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
705 int res = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
706
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
707 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
708 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
709 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
710 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
711 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
712
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
713 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
714 *exceptionObject = PyUnicodeDecodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
715 encoding, input, insize, *startinpos, *endinpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
716 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
717 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
718 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
719 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
720 if (PyUnicodeDecodeError_SetStart(*exceptionObject, *startinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
721 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
722 if (PyUnicodeDecodeError_SetEnd(*exceptionObject, *endinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
723 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
724 if (PyUnicodeDecodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
725 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
726 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
727
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
728 restuple = PyObject_CallFunctionObjArgs(*errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
729 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
730 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
731 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
732 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
733 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
734 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
735 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type, &repunicode, &newpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
736 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
737 if (newpos<0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
738 newpos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
739 else if (newpos>insize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
740 newpos = insize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
741
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
742 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
743 have+the replacement+the rest of the string (starting
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
744 at the new input position), so we won't have to check space
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
745 when there are no errors in the rest of the string) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
746 repptr = PyUnicode_AS_UNICODE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
747 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
748 requiredsize = *outpos + repsize + insize-newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
749 if (requiredsize > outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
750 if (requiredsize<2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
751 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
752 if (PyUnicode_Resize(output, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
753 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
754 *outptr = PyUnicode_AS_UNICODE(*output) + *outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
755 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
756 *endinpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
757 *inptr = input + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
758 Py_UNICODE_COPY(*outptr, repptr, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
759 *outptr += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
760 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
761 /* we made it! */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
762 res = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
763
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
764 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
765 Py_XDECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
766 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
767 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
768
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
769 /* --- UTF-7 Codec -------------------------------------------------------- */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
770
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
771 /* see RFC2152 for details */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
772
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
773 static
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
774 char utf7_special[128] = {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
775 /* indicate whether a UTF-7 character is special i.e. cannot be directly
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
776 encoded:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
777 0 - not special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
778 1 - special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
779 2 - whitespace (optional)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
780 3 - RFC2152 Set O (optional) */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
781 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
782 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
783 2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
784 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
785 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
786 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
787 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
788 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
789
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
790 };
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
791
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
792 #define SPECIAL(c, encodeO, encodeWS) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
793 (((c)>127 || utf7_special[(c)] == 1) || \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
794 (encodeWS && (utf7_special[(c)] == 2)) || \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
795 (encodeO && (utf7_special[(c)] == 3)))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
796
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
797 #define B64(n) ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[(n) & 0x3f])
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
798 #define B64CHAR(c) (isalnum(c) || (c) == '+' || (c) == '/')
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
799 #define UB64(c) ((c) == '+' ? 62 : (c) == '/' ? 63 : (c) >= 'a' ? \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
800 (c) - 71 : (c) >= 'A' ? (c) - 65 : (c) + 4)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
801
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
802 #define ENCODE(out, ch, bits) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
803 while (bits >= 6) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
804 *out++ = B64(ch >> (bits-6)); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
805 bits -= 6; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
806 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
807
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
808 #define DECODE(out, ch, bits, surrogate) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
809 while (bits >= 16) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
810 Py_UNICODE outCh = (Py_UNICODE) ((ch >> (bits-16)) & 0xffff); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
811 bits -= 16; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
812 if (surrogate) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
813 /* We have already generated an error for the high surrogate
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
814 so let's not bother seeing if the low surrogate is correct or not */\
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
815 surrogate = 0; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
816 } else if (0xDC00 <= outCh && outCh <= 0xDFFF) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
817 /* This is a surrogate pair. Unfortunately we can't represent \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
818 it in a 16-bit character */ \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
819 surrogate = 1; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
820 errmsg = "code pairs are not supported"; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
821 goto utf7Error; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
822 } else { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
823 *out++ = outCh; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
824 } \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
825 } \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
826
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
827 PyObject *PyUnicode_DecodeUTF7(const char *s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
828 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
829 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
830 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
831 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
832 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
833 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
834 int outpos;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
835 const char *e;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
836 PyUnicodeObject *unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
837 Py_UNICODE *p;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
838 const char *errmsg = "";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
839 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
840 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
841 unsigned long charsleft = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
842 int surrogate = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
843 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
844 PyObject *exc = NULL;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
845
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
846 unicode = _PyUnicode_New(size);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
847 if (!unicode)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
848 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
849 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
850 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
851
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
852 p = unicode->str;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
853 e = s + size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
854
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
855 while (s < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
856 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
857 restart:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
858 ch = *s;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
859
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
860 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
861 if ((ch == '-') || !B64CHAR(ch)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
862 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
863 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
864
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
865 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
866 if (bitsleft >= 6) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
867 /* The shift sequence has a partial character in it. If
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
868 bitsleft < 6 then we could just classify it as padding
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
869 but that is not the case here */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
870
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
871 errmsg = "partial character in shift sequence";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
872 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
873 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
874 /* According to RFC2152 the remaining bits should be zero. We
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
875 choose to signal an error/insert a replacement character
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
876 here so indicate the potential of a misencoded character. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
877
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
878 /* On x86, a << b == a << (b%32) so make sure that bitsleft != 0 */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
879 if (bitsleft && charsleft << (sizeof(charsleft) * 8 - bitsleft)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
880 errmsg = "non-zero padding bits in shift sequence";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
881 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
882 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
883
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
884 if (ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
885 if ((s < e) && (*(s) == '-')) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
886 *p++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
887 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
888 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
889 } else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
890 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
891 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
892 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
893 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
894 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
895 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
896 charsleft = (charsleft << 6) | UB64(ch);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
897 bitsleft += 6;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
898 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
899 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
900 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
901 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
902 else if ( ch == '+' ) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
903 startinpos = s-starts;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
904 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
905 if (s < e && *s == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
906 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
907 *p++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
908 } else
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
909 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
910 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
911 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
912 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
913 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
914 else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
915 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
916 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
917 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
918 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
919 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
920 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
921 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
922 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
923 continue;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
924 utf7Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
925 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
926 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
927 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
928 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
929 "utf7", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
930 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
931 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
932 goto onError;
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
933 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
934
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
935 if (inShift) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
936 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
937 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
938 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
939 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
940 "utf7", "unterminated shift sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
941 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
942 (PyObject **)&unicode, &outpos, &p))
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
943 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
944 if (s < e)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
945 goto restart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
946 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
947
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
948 if (_PyUnicode_Resize(&unicode, p - PyUnicode_AS_UNICODE(unicode)))
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
949 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
950
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
951 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
952 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
953 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
954
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
955 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
956 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
957 Py_XDECREF(exc);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
958 Py_DECREF(unicode);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
959 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
960 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
961
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
962
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
963 PyObject *PyUnicode_EncodeUTF7(const Py_UNICODE *s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
964 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
965 int encodeSetO,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
966 int encodeWhiteSpace,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
967 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
968 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
969 PyObject *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
970 /* It might be possible to tighten this worst case */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
971 unsigned int cbAllocated = 5 * size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
972 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
973 int i = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
974 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
975 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
976 char * out;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
977 char * start;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
978
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
979 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
980 return PyString_FromStringAndSize(NULL, 0);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
981
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
982 v = PyString_FromStringAndSize(NULL, cbAllocated);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
983 if (v == NULL)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
984 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
985
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
986 start = out = PyString_AS_STRING(v);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
987 for (;i < size; ++i) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
988 Py_UNICODE ch = s[i];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
989
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
990 if (!inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
991 if (ch == '+') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
992 *out++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
993 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
994 } else if (SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
995 charsleft = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
996 bitsleft = 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
997 *out++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
998 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
999 inShift = bitsleft > 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1000 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1001 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1002 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1003 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1004 if (!SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1005 *out++ = B64(charsleft << (6-bitsleft));
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1006 charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1007 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1008 /* Characters not in the BASE64 set implicitly unshift the sequence
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1009 so no '-' is required, except if the character is itself a '-' */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1010 if (B64CHAR(ch) || ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1011 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1012 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1013 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1014 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1015 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1016 bitsleft += 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1017 charsleft = (charsleft << 16) | ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1018 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1019
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1020 /* If the next character is special then we dont' need to terminate
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1021 the shift sequence. If the next character is not a BASE64 character
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1022 or '-' then the shift sequence will be terminated implicitly and we
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1023 don't have to insert a '-'. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1024
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1025 if (bitsleft == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1026 if (i + 1 < size) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1027 Py_UNICODE ch2 = s[i+1];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1028
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1029 if (SPECIAL(ch2, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1030
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1031 } else if (B64CHAR(ch2) || ch2 == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1032 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1033 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1034 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1035 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1036 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1037
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1038 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1039 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1040 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1041 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1042 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1043 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1044 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1045 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1046 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1047 if (bitsleft) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1048 *out++= B64(charsleft << (6-bitsleft) );
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1049 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1050 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1051
23295
6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people
Tim Peters <tim.peters@gmail.com>
parents: 23294
diff changeset
1052 _PyString_Resize(&v, out - start);
19998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1053 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1054 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1055
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1056 #undef SPECIAL
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1057 #undef B64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1058 #undef B64CHAR
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1059 #undef UB64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1060 #undef ENCODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1061 #undef DECODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan.
Marc-André Lemburg <mal@egenix.com>
parents: 19886
diff changeset
1062
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1063 /* --- UTF-8 Codec -------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1064
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1065 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1066 char utf8_code_length[256] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1067 /* Map UTF-8 encoded prefix byte to sequence length. zero means
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1068 illegal prefix. see RFC 2279 for details */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1069 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1070 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1071 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1072 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1073 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1074 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1075 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1076 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1077 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1078 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1079 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1080 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1081 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1082 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1083 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1084 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1085 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1086
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1087 PyObject *PyUnicode_DecodeUTF8(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1088 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1089 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1090 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1091 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1092 int n;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1093 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1094 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1095 int outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1096 const char *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1097 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1098 Py_UNICODE *p;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1099 const char *errmsg = "";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1100 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1101 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1102
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1103 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1104 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1105 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1106 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1107 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1108 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1109 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1110
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1111 /* Unpack UTF-8 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1112 p = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1113 e = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1114
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1115 while (s < e) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1116 Py_UCS4 ch = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1117
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1118 if (ch < 0x80) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1119 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1120 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1121 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1122 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1123
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1124 n = utf8_code_length[ch];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1125
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1126 if (s + n > e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1127 errmsg = "unexpected end of data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1128 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1129 endinpos = size;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1130 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1131 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1132
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1133 switch (n) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1134
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1135 case 0:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1136 errmsg = "unexpected code byte";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1137 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1138 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1139 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1140
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1141 case 1:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1142 errmsg = "internal error";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1143 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1144 endinpos = startinpos+1;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1145 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1146
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1147 case 2:
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1148 if ((s[1] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1149 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1150 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1151 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1152 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1153 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1154 ch = ((s[0] & 0x1f) << 6) + (s[1] & 0x3f);
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1155 if (ch < 0x80) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1156 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1157 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1158 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1159 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1160 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1161 else
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1162 *p++ = (Py_UNICODE)ch;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1163 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1164
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1165 case 3:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1166 if ((s[1] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1167 (s[2] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1168 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1169 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1170 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1171 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1172 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1173 ch = ((s[0] & 0x0f) << 12) + ((s[1] & 0x3f) << 6) + (s[2] & 0x3f);
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1174 if (ch < 0x0800) {
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1175 /* Note: UTF-8 encodings of surrogates are considered
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1176 legal UTF-8 sequences;
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1177
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1178 XXX For wide builds (UCS-4) we should probably try
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1179 to recombine the surrogates into a single code
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1180 unit.
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1181 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1182 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1183 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1184 endinpos = startinpos+3;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1185 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1186 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1187 else
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1188 *p++ = (Py_UNICODE)ch;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1189 break;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1190
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1191 case 4:
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1192 if ((s[1] & 0xc0) != 0x80 ||
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1193 (s[2] & 0xc0) != 0x80 ||
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1194 (s[3] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1195 errmsg = "invalid data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1196 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1197 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1198 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1199 }
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1200 ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1201 ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1202 /* validate and convert to UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1203 if ((ch < 0x10000) /* minimum value allowed for 4
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1204 byte encoding */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1205 || (ch > 0x10ffff)) /* maximum value allowed for
21886
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings.
Marc-André Lemburg <mal@egenix.com>
parents: 21884
diff changeset
1206 UTF-16 */
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1207 {
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1208 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1209 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1210 endinpos = startinpos+4;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1211 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1212 }
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1213 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1214 *p++ = (Py_UNICODE)ch;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1215 #else
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1216 /* compute and append the two surrogates: */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1217
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1218 /* translate from 10000..10FFFF to 0..FFFF */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1219 ch -= 0x10000;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1220
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1221 /* high surrogate = top 10 bits added to D800 */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1222 *p++ = (Py_UNICODE)(0xD800 + (ch >> 10));
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1223
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1224 /* low surrogate = bottom 10 bits added to DC00 */
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
1225 *p++ = (Py_UNICODE)(0xDC00 + (ch & 0x03FF));
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1226 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1227 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1228
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1229 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1230 /* Other sizes are only needed for UCS-4 */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1231 errmsg = "unsupported Unicode code range";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1232 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1233 endinpos = startinpos+n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1234 goto utf8Error;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1235 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1236 s += n;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1237 continue;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1238
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1239 utf8Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1240 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1241 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1242 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1243 "utf8", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1244 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1245 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1246 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1247 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1248
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1249 /* Adjust length */
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
1250 if (_PyUnicode_Resize(&unicode, p - unicode->str))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1251 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1252
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1253 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1254 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1255 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1256
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1257 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1258 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1259 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1260 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1261 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1262 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1263
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1264 /* Allocation strategy: if the string is short, convert into a stack buffer
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1265 and allocate exactly as much space needed at the end. Else allocate the
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1266 maximum possible needed (4 result bytes per Unicode character), and return
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1267 the excess memory at the end.
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1268 */
23198
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1269 PyObject *
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1270 PyUnicode_EncodeUTF8(const Py_UNICODE *s,
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1271 int size,
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two
Tim Peters <tim.peters@gmail.com>
parents: 23192
diff changeset
1272 const char *errors)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1273 {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1274 #define MAX_SHORT_UNICHARS 300 /* largest size we'll do on the stack */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1275
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1276 int i; /* index into s of next input byte */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1277 PyObject *v; /* result string object */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1278 char *p; /* next free byte in output buffer */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1279 int nallocated; /* number of result bytes allocated */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1280 int nneeded; /* number of result bytes needed */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1281 char stackbuf[MAX_SHORT_UNICHARS * 4];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1282
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1283 assert(s != NULL);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1284 assert(size >= 0);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1285
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1286 if (size <= MAX_SHORT_UNICHARS) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1287 /* Write into the stack buffer; nallocated can't overflow.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1288 * At the end, we'll allocate exactly as much heap space as it
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1289 * turns out we need.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1290 */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1291 nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1292 v = NULL; /* will allocate after we're done */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1293 p = stackbuf;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1294 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1295 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1296 /* Overallocate on the heap, and give the excess back at the end. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1297 nallocated = size * 4;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1298 if (nallocated / 4 != size) /* overflow! */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1299 return PyErr_NoMemory();
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1300 v = PyString_FromStringAndSize(NULL, nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1301 if (v == NULL)
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1302 return NULL;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1303 p = PyString_AS_STRING(v);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1304 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1305
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1306 for (i = 0; i < size;) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1307 Py_UCS4 ch = s[i++];
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1308
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1309 if (ch < 0x80)
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1310 /* Encode ASCII */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1311 *p++ = (char) ch;
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1312
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1313 else if (ch < 0x0800) {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1314 /* Encode Latin-1 */
21884
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1315 *p++ = (char)(0xc0 | (ch >> 6));
04cf4e13e5c9 Cosmetics.
Marc-André Lemburg <mal@egenix.com>
parents: 21883
diff changeset
1316 *p++ = (char)(0x80 | (ch & 0x3f));
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1317 }
21882
94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding
Marc-André Lemburg <mal@egenix.com>
parents: 21262
diff changeset
1318 else {
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1319 /* Encode UCS2 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1320 if (ch < 0x10000) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1321 /* Special case: check for high surrogate */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1322 if (0xD800 <= ch && ch <= 0xDBFF && i != size) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1323 Py_UCS4 ch2 = s[i];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1324 /* Check for low surrogate and combine the two to
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1325 form a UCS4 value */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1326 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1327 ch = ((ch - 0xD800) << 10 | (ch2 - 0xDC00)) + 0x10000;
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1328 i++;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1329 goto encodeUCS4;
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1330 }
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1331 /* Fall through: handles isolated high surrogates */
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1332 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
1333 *p++ = (char)(0xe0 | (ch >> 12));
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1334 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1335 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1336 continue;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1337 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1338 encodeUCS4:
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1339 /* Encode UCS4 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1340 *p++ = (char)(0xf0 | (ch >> 18));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1341 *p++ = (char)(0x80 | ((ch >> 12) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1342 *p++ = (char)(0x80 | ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1343 *p++ = (char)(0x80 | (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1344 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1345 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1346
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1347 if (v == NULL) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1348 /* This was stack allocated. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1349 nneeded = Py_SAFE_DOWNCAST(p - stackbuf, long, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1350 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1351 v = PyString_FromStringAndSize(stackbuf, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1352 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1353 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1354 /* Cut back to size actually needed. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1355 nneeded = Py_SAFE_DOWNCAST(p - PyString_AS_STRING(v), long, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1356 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1357 _PyString_Resize(&v, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1358 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1359 return v;
23203
583c226afde6 Back out 2.140.
Martin v. Löwis <martin@v.loewis.de>
parents: 23198
diff changeset
1360
23294
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8
Tim Peters <tim.peters@gmail.com>
parents: 23228
diff changeset
1361 #undef MAX_SHORT_UNICHARS
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1362 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1363
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1364 PyObject *PyUnicode_AsUTF8String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1365 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1366 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1367 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1368 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1369 }
14059
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1370 return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1371 PyUnicode_GET_SIZE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since
Barry Warsaw <barry@python.org>
parents: 13966
diff changeset
1372 NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1373 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1374
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1375 /* --- UTF-16 Codec ------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1376
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1377 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1378 PyUnicode_DecodeUTF16(const char *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1379 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1380 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1381 int *byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1382 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1383 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1384 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1385 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1386 int outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1387 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1388 Py_UNICODE *p;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1389 const unsigned char *q, *e;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1390 int bo = 0; /* assume native ordering by default */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1391 const char *errmsg = "";
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1392 /* Offsets from q for retrieving byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1393 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1394 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1395 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1396 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1397 #endif
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1398 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1399 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1400
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1401 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1402 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1403 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1404 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1405 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1406 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1407 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1408
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1409 /* Unpack UTF-16 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1410 p = unicode->str;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1411 q = (unsigned char *)s;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1412 e = q + size;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1413
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1414 if (byteorder)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1415 bo = *byteorder;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1416
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1417 /* Check for BOM marks (U+FEFF) in the input and adjust current
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1418 byte order setting accordingly. In native mode, the leading BOM
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1419 mark is skipped, in all other modes, it is copied to the output
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1420 stream as-is (giving a ZWNBSP character). */
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1421 if (bo == 0) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1422 const Py_UNICODE bom = (q[ihi] << 8) | q[ilo];
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1423 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1424 if (bom == 0xFEFF) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1425 q += 2;
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1426 bo = -1;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1427 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1428 else if (bom == 0xFFFE) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1429 q += 2;
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1430 bo = 1;
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1431 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1432 #else
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1433 if (bom == 0xFEFF) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1434 q += 2;
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1435 bo = 1;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1436 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1437 else if (bom == 0xFFFE) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1438 q += 2;
17887
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1439 bo = -1;
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1440 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1441 #endif
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the
Marc-André Lemburg <mal@egenix.com>
parents: 17700
diff changeset
1442 }
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1443
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1444 if (bo == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1445 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1446 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1447 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1448 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1449 else if (bo == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1450 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1451 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1452 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1453 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1454
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1455 while (q < e) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1456 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1457 /* remaing bytes at the end? (size should be even) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1458 if (e-q<2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1459 errmsg = "truncated data";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1460 startinpos = ((const char *)q)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1461 endinpos = ((const char *)e)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1462 goto utf16Error;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1463 /* The remaining input chars are ignored if the callback
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1464 chooses to skip the input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1465 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1466 ch = (q[ihi] << 8) | q[ilo];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1467
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1468 q += 2;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1469
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1470 if (ch < 0xD800 || ch > 0xDFFF) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1471 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1472 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1473 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1474
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1475 /* UTF-16 code pair: */
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1476 if (q >= e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1477 errmsg = "unexpected end of data";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1478 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1479 endinpos = ((const char *)e)-starts;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1480 goto utf16Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1481 }
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
1482 if (0xD800 <= ch && ch <= 0xDBFF) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1483 Py_UNICODE ch2 = (q[ihi] << 8) | q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1484 q += 2;
18295
20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness
Martin v. Löwis <martin@v.loewis.de>
parents: 18292
diff changeset
1485 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1486 #ifndef Py_UNICODE_WIDE
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1487 *p++ = ch;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1488 *p++ = ch2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1489 #else
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1490 *p++ = (((ch & 0x3FF)<<10) | (ch2 & 0x3FF)) + 0x10000;
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1491 #endif
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1492 continue;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1493 }
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1494 else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1495 errmsg = "illegal UTF-16 surrogate";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1496 startinpos = (((const char *)q)-4)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1497 endinpos = startinpos+2;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1498 goto utf16Error;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1499 }
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1500
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1501 }
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1502 errmsg = "illegal encoding";
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1503 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1504 endinpos = startinpos+2;
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1505 /* Fall through to report the error */
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1506
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1507 utf16Error:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1508 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1509 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1510 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1511 "utf16", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1512 starts, size, &startinpos, &endinpos, &exc, (const char **)&q,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1513 (PyObject **)&unicode, &outpos, &p))
13635
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs.
Marc-André Lemburg <mal@egenix.com>
parents: 13633
diff changeset
1514 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1515 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1516
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1517 if (byteorder)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1518 *byteorder = bo;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1519
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1520 /* Adjust length */
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
1521 if (_PyUnicode_Resize(&unicode, p - unicode->str))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1522 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1523
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1524 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1525 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1526 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1527
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1528 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1529 Py_DECREF(unicode);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1530 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1531 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1532 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1533 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1534
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1535 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1536 PyUnicode_EncodeUTF16(const Py_UNICODE *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1537 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1538 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1539 int byteorder)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1540 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1541 PyObject *v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1542 unsigned char *p;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1543 int i, pairs;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1544 /* Offsets from p for storing byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1545 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1546 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1547 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1548 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1549 #endif
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1550
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1551 #define STORECHAR(CH) \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1552 do { \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1553 p[ihi] = ((CH) >> 8) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1554 p[ilo] = (CH) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1555 p += 2; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1556 } while(0)
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1557
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1558 for (i = pairs = 0; i < size; i++)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1559 if (s[i] >= 0x10000)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1560 pairs++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1561 v = PyString_FromStringAndSize(NULL,
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1562 2 * (size + pairs + (byteorder == 0)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1563 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1564 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1565
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1566 p = (unsigned char *)PyString_AS_STRING(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1567 if (byteorder == 0)
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1568 STORECHAR(0xFEFF);
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
1569 if (size == 0)
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
1570 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1571
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1572 if (byteorder == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1573 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1574 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1575 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1576 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1577 else if (byteorder == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1578 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1579 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1580 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1581 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1582
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1583 while (size-- > 0) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1584 Py_UNICODE ch = *s++;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1585 Py_UNICODE ch2 = 0;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1586 if (ch >= 0x10000) {
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1587 ch2 = 0xDC00 | ((ch-0x10000) & 0x3FF);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1588 ch = 0xD800 | ((ch-0x10000) >> 10);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1589 }
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1590 STORECHAR(ch);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1591 if (ch2)
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1592 STORECHAR(ch2);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1593 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1594 return v;
19121
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions
Tim Peters <tim.peters@gmail.com>
parents: 18933
diff changeset
1595 #undef STORECHAR
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1596 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1597
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1598 PyObject *PyUnicode_AsUTF16String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1599 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1600 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1601 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1602 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1603 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1604 return PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1605 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1606 NULL,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1607 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1608 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1609
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1610 /* --- Unicode Escape Codec ----------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1611
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1612 static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL;
13063
eb243443b884 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13049
diff changeset
1613
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1614 PyObject *PyUnicode_DecodeUnicodeEscape(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1615 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1616 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1617 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1618 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1619 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1620 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1621 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1622 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1623 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1624 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1625 const char *end;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1626 char* message;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1627 Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1628 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1629 PyObject *exc = NULL;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1630
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1631 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1632 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1633 length after conversion to the true value.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1634 (but if the error callback returns a long replacement string
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1635 we'll have to allocate more space) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1636 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1637 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1638 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1639 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1640 return (PyObject *)v;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1641
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1642 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1643 end = s + size;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1644
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1645 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1646 unsigned char c;
13396
759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and
Marc-André Lemburg <mal@egenix.com>
parents: 13394
diff changeset
1647 Py_UNICODE x;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1648 int digits;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1649
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1650 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1651 if (*s != '\\') {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1652 *p++ = (unsigned char) *s++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1653 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1654 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1655
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1656 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1657 /* \ - Escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1658 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1659 switch (*s++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1660
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1661 /* \x escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1662 case '\n': break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1663 case '\\': *p++ = '\\'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1664 case '\'': *p++ = '\''; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1665 case '\"': *p++ = '\"'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1666 case 'b': *p++ = '\b'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1667 case 'f': *p++ = '\014'; break; /* FF */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1668 case 't': *p++ = '\t'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1669 case 'n': *p++ = '\n'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1670 case 'r': *p++ = '\r'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1671 case 'v': *p++ = '\013'; break; /* VT */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1672 case 'a': *p++ = '\007'; break; /* BEL, not classic C */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1673
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1674 /* \OOO (octal) escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1675 case '0': case '1': case '2': case '3':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1676 case '4': case '5': case '6': case '7':
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1677 x = s[-1] - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1678 if ('0' <= *s && *s <= '7') {
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1679 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1680 if ('0' <= *s && *s <= '7')
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1681 x = (x<<3) + *s++ - '0';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1682 }
12544
599bfa4ffd49 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12511
diff changeset
1683 *p++ = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1684 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1685
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1686 /* hex escapes */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1687 /* \xXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1688 case 'x':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1689 digits = 2;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1690 message = "truncated \\xXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1691 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1692
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1693 /* \uXXXX */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1694 case 'u':
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1695 digits = 4;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1696 message = "truncated \\uXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1697 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1698
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1699 /* \UXXXXXXXX */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1700 case 'U':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1701 digits = 8;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1702 message = "truncated \\UXXXXXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1703 hexescape:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1704 chr = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1705 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1706 if (s+digits>end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1707 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1708 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1709 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1710 "unicodeescape", "end of string in escape sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1711 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1712 (PyObject **)&v, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1713 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1714 goto nextByte;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1715 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1716 for (i = 0; i < digits; ++i) {
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1717 c = (unsigned char) s[i];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1718 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1719 endinpos = (s+i+1)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1720 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1721 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1722 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1723 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1724 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1725 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1726 goto nextByte;
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1727 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1728 chr = (chr<<4) & ~0xF;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1729 if (c >= '0' && c <= '9')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1730 chr += c - '0';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1731 else if (c >= 'a' && c <= 'f')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1732 chr += 10 + c - 'a';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1733 else
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1734 chr += 10 + c - 'A';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1735 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1736 s += i;
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1737 if (chr == 0xffffffff)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1738 /* _decoding_error will have already written into the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1739 target buffer. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1740 break;
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1741 store:
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1742 /* when we get here, chr is a 32-bit unicode character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1743 if (chr <= 0xffff)
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1744 /* UCS-2 character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1745 *p++ = (Py_UNICODE) chr;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1746 else if (chr <= 0x10ffff) {
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1747 /* UCS-4 character. Either store directly, or as
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1748 surrogate pair. */
18324
3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE
Fredrik Lundh <fredrik@pythonware.com>
parents: 18298
diff changeset
1749 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1750 *p++ = chr;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1751 #else
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1752 chr -= 0x10000L;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1753 *p++ = 0xD800 + (Py_UNICODE) (chr >> 10);
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
1754 *p++ = 0xDC00 + (Py_UNICODE) (chr & 0x03FF);
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1755 #endif
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1756 } else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1757 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1758 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1759 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1760 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1761 "unicodeescape", "illegal Unicode character",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1762 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1763 (PyObject **)&v, &outpos, &p))
14388
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1764 goto onError;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode
Fredrik Lundh <fredrik@pythonware.com>
parents: 14067
diff changeset
1765 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1766 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1767
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1768 /* \N{name} */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1769 case 'N':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1770 message = "malformed \\N character escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1771 if (ucnhash_CAPI == NULL) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1772 /* load the unicode data module */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1773 PyObject *m, *v;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1774 m = PyImport_ImportModule("unicodedata");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1775 if (m == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1776 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1777 v = PyObject_GetAttrString(m, "ucnhash_CAPI");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1778 Py_DECREF(m);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1779 if (v == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1780 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1781 ucnhash_CAPI = PyCObject_AsVoidPtr(v);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1782 Py_DECREF(v);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1783 if (ucnhash_CAPI == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1784 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1785 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1786 if (*s == '{') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1787 const char *start = s+1;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1788 /* look for the closing brace */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1789 while (*s != '}' && s < end)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1790 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1791 if (s > start && s < end && *s == '}') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1792 /* found a name. look it up in the unicode database */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1793 message = "unknown Unicode character name";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1794 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1795 if (ucnhash_CAPI->getcode(start, s-start-1, &chr))
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1796 goto store;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1797 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1798 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1799 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1800 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1801 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1802 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1803 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1804 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1805 (PyObject **)&v, &outpos, &p))
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1806 goto onError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1807 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1808
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1809 default:
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1810 if (s > end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1811 message = "\\ at end of string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1812 s--;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1813 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1814 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1815 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1816 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1817 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1818 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1819 (PyObject **)&v, &outpos, &p))
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1820 goto onError;
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1821 }
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1822 else {
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1823 *p++ = '\\';
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1824 *p++ = (unsigned char)s[-1];
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1825 }
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1826 break;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1827 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1828 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1829 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1830 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1831 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1832 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1833 return (PyObject *)v;
22521
795bdd692176 Fix whitespace.
Walter Dörwald <walter@livinglogic.de>
parents: 22446
diff changeset
1834
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1835 ucnhashError:
16200
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1836 PyErr_SetString(
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1837 PyExc_UnicodeError,
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1838 "\\N escapes not supported (can't load unicodedata module)"
3484670c37f2 Move uchhash functionality into unicodedata (after the recent
Fredrik Lundh <fredrik@pythonware.com>
parents: 16064
diff changeset
1839 );
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1840 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1841 Py_XDECREF(exc);
16064
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
1842 return NULL;
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute
Fredrik Lundh <fredrik@pythonware.com>
parents: 16029
diff changeset
1843
16667
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it
Fredrik Lundh <fredrik@pythonware.com>
parents: 16332
diff changeset
1844 onError:
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1845 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1846 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
1847 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1848 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1849 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1850
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1851 /* Return a Unicode-Escape string version of the Unicode object.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1852
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1853 If quotes is true, the string is enclosed in u"" or u'' quotes as
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1854 appropriate.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1855
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1856 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1857
12025
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1858 static const Py_UNICODE *findchar(const Py_UNICODE *s,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1859 int size,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1860 Py_UNICODE ch);
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said:
Barry Warsaw <barry@python.org>
parents: 11997
diff changeset
1861
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1862 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1863 PyObject *unicodeescape_string(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1864 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1865 int quotes)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1866 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1867 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1868 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1869
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1870 static const char *hexdigit = "0123456789abcdef";
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1871
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1872 repr = PyString_FromStringAndSize(NULL, 2 + 6*size + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1873 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1874 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1875
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1876 p = PyString_AS_STRING(repr);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1877
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1878 if (quotes) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1879 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1880 *p++ = (findchar(s, size, '\'') &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1881 !findchar(s, size, '"')) ? '"' : '\'';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1882 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1883 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1884 Py_UNICODE ch = *s++;
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1885
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1886 /* Escape quotes */
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1887 if (quotes &&
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1888 (ch == (Py_UNICODE) PyString_AS_STRING(repr)[1] || ch == '\\')) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1889 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1890 *p++ = (char) ch;
20018
8aa1ada33f64 Fix a bug in rendering of \\ by repr() -- it rendered as \\\ instead
Guido van Rossum <guido@python.org>
parents: 20007
diff changeset
1891 continue;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1892 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1893
18773
743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This
Guido van Rossum <guido@python.org>
parents: 18324
diff changeset
1894 #ifdef Py_UNICODE_WIDE
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1895 /* Map 21-bit characters to '\U00xxxxxx' */
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1896 else if (ch >= 0x10000) {
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1897 int offset = p - PyString_AS_STRING(repr);
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1898
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1899 /* Resize the string if necessary */
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1900 if (offset + 12 > PyString_GET_SIZE(repr)) {
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1901 if (_PyString_Resize(&repr, PyString_GET_SIZE(repr) + 100))
23295
6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people
Tim Peters <tim.peters@gmail.com>
parents: 23294
diff changeset
1902 return NULL;
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1903 p = PyString_AS_STRING(repr) + offset;
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1904 }
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1905
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1906 *p++ = '\\';
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1907 *p++ = 'U';
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1908 *p++ = hexdigit[(ch >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1909 *p++ = hexdigit[(ch >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1910 *p++ = hexdigit[(ch >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1911 *p++ = hexdigit[(ch >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1912 *p++ = hexdigit[(ch >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1913 *p++ = hexdigit[(ch >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1914 *p++ = hexdigit[(ch >> 4) & 0x0000000F];
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1915 *p++ = hexdigit[ch & 0x0000000F];
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1916 continue;
18292
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type:
Martin v. Löwis <martin@v.loewis.de>
parents: 18280
diff changeset
1917 }
18773
743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This
Guido van Rossum <guido@python.org>
parents: 18324
diff changeset
1918 #endif
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1919 /* Map UTF-16 surrogate pairs to Unicode \UXXXXXXXX escapes */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1920 else if (ch >= 0xD800 && ch < 0xDC00) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1921 Py_UNICODE ch2;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1922 Py_UCS4 ucs;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1923
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1924 ch2 = *s++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1925 size--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1926 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1927 ucs = (((ch & 0x03FF) << 10) | (ch2 & 0x03FF)) + 0x00010000;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1928 *p++ = '\\';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1929 *p++ = 'U';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1930 *p++ = hexdigit[(ucs >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1931 *p++ = hexdigit[(ucs >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1932 *p++ = hexdigit[(ucs >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1933 *p++ = hexdigit[(ucs >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1934 *p++ = hexdigit[(ucs >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1935 *p++ = hexdigit[(ucs >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1936 *p++ = hexdigit[(ucs >> 4) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1937 *p++ = hexdigit[ucs & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1938 continue;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1939 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1940 /* Fall through: isolated surrogates are copied as-is */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1941 s--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1942 size++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1943 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1944
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1945 /* Map 16-bit characters to '\uxxxx' */
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1946 if (ch >= 256) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1947 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1948 *p++ = 'u';
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1949 *p++ = hexdigit[(ch >> 12) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1950 *p++ = hexdigit[(ch >> 8) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1951 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1952 *p++ = hexdigit[ch & 0x000F];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1953 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1954
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1955 /* Map special whitespace to '\t', \n', '\r' */
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1956 else if (ch == '\t') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1957 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1958 *p++ = 't';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1959 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1960 else if (ch == '\n') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1961 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1962 *p++ = 'n';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1963 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1964 else if (ch == '\r') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1965 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1966 *p++ = 'r';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1967 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1968
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1969 /* Map non-printable US ASCII to '\xhh' */
21067
c9c4c523dcbf Fix for bug #485951: repr diff between string and unicode.
Marc-André Lemburg <mal@egenix.com>
parents: 21028
diff changeset
1970 else if (ch < ' ' || ch >= 0x7F) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1971 *p++ = '\\';
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
1972 *p++ = 'x';
18774
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1973 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates
Marc-André Lemburg <mal@egenix.com>
parents: 18773
diff changeset
1974 *p++ = hexdigit[ch & 0x000F];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1975 }
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1976
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1977 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1978 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1979 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1980 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1981 if (quotes)
18828
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on
Marc-André Lemburg <mal@egenix.com>
parents: 18774
diff changeset
1982 *p++ = PyString_AS_STRING(repr)[1];
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1983
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1984 *p = '\0';
23295
6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people
Tim Peters <tim.peters@gmail.com>
parents: 23294
diff changeset
1985 _PyString_Resize(&repr, p - PyString_AS_STRING(repr));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1986 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1987 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1988
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1989 PyObject *PyUnicode_EncodeUnicodeEscape(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1990 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1991 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1992 return unicodeescape_string(s, size, 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1993 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1994
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1995 PyObject *PyUnicode_AsUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1996 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1997 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1998 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
1999 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2000 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2001 return PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2002 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2003 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2004
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2005 /* --- Raw Unicode Escape Codec ------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2006
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2007 PyObject *PyUnicode_DecodeRawUnicodeEscape(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2008 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2009 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2010 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2011 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2012 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2013 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2014 int outpos;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2015 PyUnicodeObject *v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2016 Py_UNICODE *p;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2017 const char *end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2018 const char *bs;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2019 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2020 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2021
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2022 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2023 Unicode string, so we start with size here and then reduce the
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2024 length after conversion to the true value. (But decoding error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2025 handler might have to resize the string) */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2026 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2027 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2028 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2029 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2030 return (PyObject *)v;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2031 p = PyUnicode_AS_UNICODE(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2032 end = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2033 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2034 unsigned char c;
22423
b523a470a0ce Do not insert characters for unicode-escape decoders if the error mode
Martin v. Löwis <martin@v.loewis.de>
parents: 22087
diff changeset
2035 Py_UCS4 x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2036 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2037
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2038 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2039 if (*s != '\\') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2040 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2041 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2042 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2043 startinpos = s-starts;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2044
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2045 /* \u-escapes are only interpreted iff the number of leading
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2046 backslashes if odd */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2047 bs = s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2048 for (;s < end;) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2049 if (*s != '\\')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2050 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2051 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2052 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2053 if (((s - bs) & 1) == 0 ||
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2054 s >= end ||
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2055 *s != 'u') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2056 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2057 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2058 p--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2059 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2060
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2061 /* \uXXXX with 4 hex digits */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2062 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2063 for (x = 0, i = 0; i < 4; ++i, ++s) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2064 c = (unsigned char)*s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2065 if (!isxdigit(c)) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2066 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2067 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2068 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2069 "rawunicodeescape", "truncated \\uXXXX",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2070 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2071 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2072 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2073 goto nextByte;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2074 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2075 x = (x<<4) & ~0xF;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2076 if (c >= '0' && c <= '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2077 x += c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2078 else if (c >= 'a' && c <= 'f')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2079 x += 10 + c - 'a';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2080 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2081 x += 10 + c - 'A';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2082 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2083 *p++ = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2084 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2085 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2086 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2087 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2088 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2089 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2090 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2091 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2092
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2093 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2094 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2095 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2096 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2097 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2098 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2099
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2100 PyObject *PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2101 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2102 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2103 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2104 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2105 char *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2106
16226
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings.
Ka-Ping Yee <ping@zesty.ca>
parents: 16200
diff changeset
2107 static const char *hexdigit = "0123456789abcdef";
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2108
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2109 repr = PyString_FromStringAndSize(NULL, 6 * size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2110 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2111 return NULL;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2112 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2113 return repr;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2114
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2115 p = q = PyString_AS_STRING(repr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2116 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2117 Py_UNICODE ch = *s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2118 /* Map 16-bit characters to '\uxxxx' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2119 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2120 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2121 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2122 *p++ = hexdigit[(ch >> 12) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2123 *p++ = hexdigit[(ch >> 8) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2124 *p++ = hexdigit[(ch >> 4) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2125 *p++ = hexdigit[ch & 15];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2126 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2127 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2128 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2129 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2130 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2131 *p = '\0';
23295
6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people
Tim Peters <tim.peters@gmail.com>
parents: 23294
diff changeset
2132 _PyString_Resize(&repr, p - q);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2133 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2134 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2135
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2136 PyObject *PyUnicode_AsRawUnicodeEscapeString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2137 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2138 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2139 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2140 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2141 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2142 return PyUnicode_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2143 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2144 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2145
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2146 /* --- Latin-1 Codec ------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2147
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2148 PyObject *PyUnicode_DecodeLatin1(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2149 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2150 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2151 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2152 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2153 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2154
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2155 /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2156 if (size == 1 && *(unsigned char*)s < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2157 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2158 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2159 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2160
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2161 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2162 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2163 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2164 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2165 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2166 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2167 while (size-- > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2168 *p++ = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2169 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2170
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2171 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2172 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2173 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2174 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2175
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2176 /* create or adjust a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2177 static void make_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2178 const char *encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2179 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2180 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2181 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2182 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2183 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2184 *exceptionObject = PyUnicodeEncodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2185 encoding, unicode, size, startpos, endpos, reason);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2186 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2187 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2188 if (PyUnicodeEncodeError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2189 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2190 if (PyUnicodeEncodeError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2191 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2192 if (PyUnicodeEncodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2193 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2194 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2195 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2196 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2197 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2198 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2199 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2200
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2201 /* raises a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2202 static void raise_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2203 const char *encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2204 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2205 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2206 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2207 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2208 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2209 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2210 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2211 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2212 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2213
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2214 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2215 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2216 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2217 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2218 static PyObject *unicode_encode_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2219 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2220 const char *encoding, const char *reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2221 const Py_UNICODE *unicode, int size, PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2222 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2223 int *newpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2224 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2225 static char *argparse = "O!i;encoding error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2226
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2227 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2228 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2229
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2230 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2231 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2232 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2233 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2234 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2235
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2236 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2237 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2238 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2239 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2240
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2241 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2242 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2243 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2244 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2245 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2246 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2247 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2248 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2249 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2250 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2251 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2252 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2253 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2254 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2255 if (*newpos<0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2256 *newpos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2257 else if (*newpos>size)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2258 *newpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2259 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2260 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2261 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2262 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2263
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2264 static PyObject *unicode_encode_ucs1(const Py_UNICODE *p,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2265 int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2266 const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2267 int limit)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2268 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2269 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2270 PyObject *res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2271 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2272 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2273 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2274 /* pointer to the beginning of the unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2275 /* const Py_UNICODE *badp = NULL; */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2276 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2277 char *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2278 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2279 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2280 int ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2281 char *encoding = (limit == 256) ? "latin-1" : "ascii";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2282 char *reason = (limit == 256) ? "ordinal not in range(256)" : "ordinal not in range(128)";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2283 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2284 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2285 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2286 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2287 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2288
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2289 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2290 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2291 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2292 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2293 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2294 if (size == 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2295 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2296 str = PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2297 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2298
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2299 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2300 Py_UNICODE c = *p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2301
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2302 /* can we encode this? */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2303 if (c<limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2304 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2305 *str++ = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2306 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2307 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2308 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2309 int unicodepos = p-startp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2310 int requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2311 PyObject *repunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2312 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2313 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2314 int respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2315 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2316 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2317 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2318 const Py_UNICODE *collend = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2319 /* find all unecodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2320 while ((collend < endp) && ((*collend)>=limit))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2321 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2322 /* cache callback name lookup (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2323 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2324 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2325 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2326 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2327 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2328 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2329 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2330 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2331 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2332 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2333 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2334 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2335 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2336 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2337 raise_encode_exception(&exc, encoding, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2338 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2339 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2340 while (collstart++<collend)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2341 *str++ = '?'; /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2342 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2343 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2344 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2345 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2346 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2347 /* determine replacement size (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2348 for (p = collstart, repsize = 0; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2349 if (*p<10)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2350 repsize += 2+1+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2351 else if (*p<100)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2352 repsize += 2+2+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2353 else if (*p<1000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2354 repsize += 2+3+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2355 else if (*p<10000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2356 repsize += 2+4+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2357 else if (*p<100000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2358 repsize += 2+5+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2359 else if (*p<1000000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2360 repsize += 2+6+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2361 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2362 repsize += 2+7+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2363 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2364 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2365 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2366 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2367 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2368 if (_PyString_Resize(&res, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2369 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2370 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2371 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2372 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2373 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2374 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2375 str += sprintf(str, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2376 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2377 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2378 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2379 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2380 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2381 encoding, reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2382 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2383 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2384 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2385 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2386 have+the replacement+the rest of the string, so
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2387 we won't have to check space for encodable characters) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2388 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2389 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2390 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2391 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2392 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2393 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2394 if (_PyString_Resize(&res, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2395 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2396 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2397 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2398 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2399 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2400 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2401 /* check if there is anything unencodable in the replacement
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2402 and copy it to the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2403 for (uni2 = PyUnicode_AS_UNICODE(repunicode);repsize-->0; ++uni2, ++str) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2404 c = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2405 if (c >= limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2406 raise_encode_exception(&exc, encoding, startp, size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2407 unicodepos, unicodepos+1, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2408 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2409 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2410 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2411 *str = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2412 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2413 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2414 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2415 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2416 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2417 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2418 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2419 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2420 if (respos<ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2421 /* If this falls res will be NULL */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2422 _PyString_Resize(&res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2423 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2424 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2425 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2426
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2427 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2428 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2429 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2430 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2431 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2432 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2433
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2434 PyObject *PyUnicode_EncodeLatin1(const Py_UNICODE *p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2435 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2436 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2437 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2438 return unicode_encode_ucs1(p, size, errors, 256);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2439 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2440
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2441 PyObject *PyUnicode_AsLatin1String(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2442 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2443 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2444 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2445 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2446 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2447 return PyUnicode_EncodeLatin1(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2448 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2449 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2450 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2451
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2452 /* --- 7-bit ASCII Codec -------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2453
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2454 PyObject *PyUnicode_DecodeASCII(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2455 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2456 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2457 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2458 const char *starts = s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2459 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2460 Py_UNICODE *p;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2461 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2462 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2463 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2464 const char *e;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2465 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2466 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2467
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2468 /* ASCII is equivalent to the first 128 ordinals in Unicode. */
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2469 if (size == 1 && *(unsigned char*)s < 128) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2470 Py_UNICODE r = *(unsigned char*)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2471 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2472 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2473
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2474 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2475 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2476 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2477 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2478 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2479 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2480 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2481 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2482 register unsigned char c = (unsigned char)*s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2483 if (c < 128) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2484 *p++ = c;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2485 ++s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2486 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2487 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2488 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2489 endinpos = startinpos + 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2490 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2491 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2492 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2493 "ascii", "ordinal not in range(128)",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2494 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2495 (PyObject **)&v, &outpos, &p))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2496 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2497 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2498 }
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2499 if (p - PyUnicode_AS_UNICODE(v) < PyString_GET_SIZE(v))
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2500 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
2501 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2502 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2503 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2504 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2505
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2506 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2507 Py_XDECREF(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2508 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2509 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2510 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2511 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2512
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2513 PyObject *PyUnicode_EncodeASCII(const Py_UNICODE *p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2514 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2515 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2516 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2517 return unicode_encode_ucs1(p, size, errors, 128);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2518 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2519
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2520 PyObject *PyUnicode_AsASCIIString(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2521 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2522 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2523 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2524 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2525 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2526 return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2527 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2528 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2529 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2530
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
2531 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
2532
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2533 /* --- MBCS codecs for Windows -------------------------------------------- */
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
2534
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2535 PyObject *PyUnicode_DecodeMBCS(const char *s,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2536 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2537 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2538 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2539 PyUnicodeObject *v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2540 Py_UNICODE *p;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2541
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2542 /* First get the size of the result */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2543 DWORD usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0);
12582
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2544 if (size > 0 && usize==0)
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2545 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2546
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2547 v = _PyUnicode_New(usize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2548 if (v == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2549 return NULL;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2550 if (usize == 0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2551 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2552 p = PyUnicode_AS_UNICODE(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2553 if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2554 Py_DECREF(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2555 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2556 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2557
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2558 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2559 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2560
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2561 PyObject *PyUnicode_EncodeMBCS(const Py_UNICODE *p,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2562 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2563 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2564 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2565 PyObject *repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2566 char *s;
12582
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2567 DWORD mbcssize;
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2568
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2569 /* If there are no characters, bail now! */
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2570 if (size==0)
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2571 return PyString_FromString("");
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2572
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2573 /* First get the size of the result */
12582
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length
Guido van Rossum <guido@python.org>
parents: 12577
diff changeset
2574 mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL);
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2575 if (mbcssize==0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2576 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2577
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2578 repr = PyString_FromStringAndSize(NULL, mbcssize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2579 if (repr == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2580 return NULL;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2581 if (mbcssize == 0)
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2582 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2583
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2584 /* Do the conversion */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2585 s = PyString_AS_STRING(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2586 if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2587 Py_DECREF(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2588 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2589 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2590 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2591 }
12193
c3fc2b4617fc Some blank lines.
Guido van Rossum <guido@python.org>
parents: 12067
diff changeset
2592
24071
8142f8d668a0 Patch #569753: Remove support for WIN16.
Martin v. Löwis <martin@v.loewis.de>
parents: 23837
diff changeset
2593 #endif /* MS_WINDOWS */
12067
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond.
Guido van Rossum <guido@python.org>
parents: 12025
diff changeset
2594
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2595 /* --- Character Mapping Codec -------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2596
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2597 PyObject *PyUnicode_DecodeCharmap(const char *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2598 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2599 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2600 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2601 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2602 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2603 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2604 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2605 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2606 const char *e;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2607 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2608 Py_UNICODE *p;
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2609 int extrachars = 0;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2610 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2611 PyObject *exc = NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2613 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2614 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2615 return PyUnicode_DecodeLatin1(s, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2616
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2617 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2618 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2619 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2620 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2621 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2622 p = PyUnicode_AS_UNICODE(v);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2623 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2624 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2625 unsigned char ch = *s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2626 PyObject *w, *x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2627
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2628 /* Get mapping (char ordinal -> integer, Unicode char or None) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2629 w = PyInt_FromLong((long)ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2630 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2631 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2632 x = PyObject_GetItem(mapping, w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2633 Py_DECREF(w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2634 if (x == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2635 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
15706
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2636 /* No mapping found means: mapping is undefined. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2637 PyErr_Clear();
15706
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2638 x = Py_None;
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2639 Py_INCREF(x);
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap
Marc-André Lemburg <mal@egenix.com>
parents: 15631
diff changeset
2640 } else
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2641 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2642 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2643
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2644 /* Apply mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2645 if (PyInt_Check(x)) {
13390
62754b9ec7df Fixed a couple of places where 'int' was used where 'long'
Marc-André Lemburg <mal@egenix.com>
parents: 13356
diff changeset
2646 long value = PyInt_AS_LONG(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2647 if (value < 0 || value > 65535) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2648 PyErr_SetString(PyExc_TypeError,
12909
be36f103a953 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12902
diff changeset
2649 "character mapping must be in range(65536)");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2650 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2651 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2652 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2653 *p++ = (Py_UNICODE)value;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2654 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2655 else if (x == Py_None) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2656 /* undefined mapping */
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2657 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2658 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2659 endinpos = startinpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2660 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2661 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2662 "charmap", "character maps to <undefined>",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2663 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2664 (PyObject **)&v, &outpos, &p)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2665 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2666 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2667 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2668 continue;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2669 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2670 else if (PyUnicode_Check(x)) {
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2671 int targetsize = PyUnicode_GET_SIZE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2672
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2673 if (targetsize == 1)
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2674 /* 1-1 mapping */
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2675 *p++ = *PyUnicode_AS_UNICODE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2676
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2677 else if (targetsize > 1) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2678 /* 1-n mapping */
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2679 if (targetsize > extrachars) {
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2680 /* resize first */
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2681 int oldpos = (int)(p - PyUnicode_AS_UNICODE(v));
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2682 int needed = (targetsize - extrachars) + \
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2683 (targetsize << 2);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2684 extrachars += needed;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2685 if (_PyUnicode_Resize(&v,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2686 PyUnicode_GET_SIZE(v) + needed)) {
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2687 Py_DECREF(x);
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2688 goto onError;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2689 }
15749
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2690 p = PyUnicode_AS_UNICODE(v) + oldpos;
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2691 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2692 Py_UNICODE_COPY(p,
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2693 PyUnicode_AS_UNICODE(x),
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2694 targetsize);
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2695 p += targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2696 extrachars -= targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2697 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec:
Marc-André Lemburg <mal@egenix.com>
parents: 15706
diff changeset
2698 /* 1-0 mapping: skip the character */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2699 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2700 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2701 /* wrong return value */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2702 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2703 "character mapping must return integer, None or unicode");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2704 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2705 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2706 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2707 Py_DECREF(x);
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2708 ++s;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2709 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2710 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
2711 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2712 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2713 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2714 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2715 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2716
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2717 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2718 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2719 Py_XDECREF(exc);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2720 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2721 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2722 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2723
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2724 /* Lookup the character ch in the mapping. If the character
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2725 can't be found, Py_None is returned (or NULL, if another
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2726 error occured). */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2727 static PyObject *charmapencode_lookup(Py_UNICODE c, PyObject *mapping)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2728 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2729 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2730 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2731
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2732 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2733 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2734 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2735 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2736 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2737 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2738 /* No mapping found means: mapping is undefined. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2739 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2740 x = Py_None;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2741 Py_INCREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2742 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2743 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2744 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2745 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2746 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2747 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2748 if (value < 0 || value > 255) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2749 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2750 "character mapping must be in range(256)");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2751 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2752 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2753 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2754 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2755 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2756 else if (PyString_Check(x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2757 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2758 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2759 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2760 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2761 "character mapping must return integer, None or str");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2762 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2763 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2764 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2765 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2766
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2767 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2768 various state variables. Reallocate the output string if not enough
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2769 space is available. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2770 was put in the output buffer, or Py_None, if the mapping was undefined
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2771 (in which case no character was written) or NULL, if a
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2772 reallocation error ocurred. The called must decref the result */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2773 static
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2774 PyObject *charmapencode_output(Py_UNICODE c, PyObject *mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2775 PyObject **outobj, int *outpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2776 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2777 PyObject *rep = charmapencode_lookup(c, mapping);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2778
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2779 if (rep==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2780 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2781 else if (rep==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2782 return rep;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2783 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2784 char *outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2785 int outsize = PyString_GET_SIZE(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2786 if (PyInt_Check(rep)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2787 int requiredsize = *outpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2788 if (outsize<requiredsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2789 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2790 if (requiredsize < 2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2791 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2792 if (_PyString_Resize(outobj, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2793 Py_DECREF(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2794 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2795 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2796 outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2797 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2798 outstart[(*outpos)++] = (char)PyInt_AS_LONG(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2799 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2800 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2801 const char *repchars = PyString_AS_STRING(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2802 int repsize = PyString_GET_SIZE(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2803 int requiredsize = *outpos+repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2804 if (outsize<requiredsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2805 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2806 if (requiredsize < 2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2807 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2808 if (_PyString_Resize(outobj, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2809 Py_DECREF(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2810 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2811 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2812 outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2813 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2814 memcpy(outstart + *outpos, repchars, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2815 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2816 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2817 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2818 return rep;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2819 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2820
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2821 /* handle an error in PyUnicode_EncodeCharmap
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2822 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2823 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2824 int charmap_encoding_error(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2825 const Py_UNICODE *p, int size, int *inpos, PyObject *mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2826 PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2827 int *known_errorHandler, PyObject *errorHandler, const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2828 PyObject **res, int *respos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2829 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2830 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2831 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2832 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2833 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2834 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2835 int collstartpos = *inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2836 int collendpos = *inpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2837 int collpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2838 char *encoding = "charmap";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2839 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2840
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2841 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2842 /* find all unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2843 while (collendpos < size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2844 x = charmapencode_lookup(p[collendpos], mapping);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2845 if (x==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2846 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2847 else if (x!=Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2848 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2849 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2850 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2851 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2852 ++collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2853 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2854 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2855 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2856 if (*known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2857 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2858 *known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2859 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2860 *known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2861 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2862 *known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2863 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2864 *known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2865 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2866 *known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2867 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2868 switch (*known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2869 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2870 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2871 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2872 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2873 for (collpos = collstartpos; collpos<collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2874 x = charmapencode_output('?', mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2875 if (x==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2876 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2877 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2878 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2879 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2880 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2881 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2882 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2883 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2884 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2885 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2886 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2887 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2888 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2889 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2890 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2891 for (collpos = collstartpos; collpos < collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2892 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2893 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2894 sprintf(buffer, "&#%d;", (int)p[collpos]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2895 for (cp = buffer; *cp; ++cp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2896 x = charmapencode_output(*cp, mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2897 if (x==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2898 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2899 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2900 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2901 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2902 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2903 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2904 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2905 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2906 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2907 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2908 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2909 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2910 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2911 encoding, reason, p, size, exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2912 collstartpos, collendpos, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2913 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2914 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2915 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2916 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2917 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2918 x = charmapencode_output(*uni2, mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2919 if (x==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2920 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2921 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2922 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2923 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2924 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2925 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2926 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2927 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2928 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2929 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2930 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2931 *inpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2932 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2933 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2934 return 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2935 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2936
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2937 PyObject *PyUnicode_EncodeCharmap(const Py_UNICODE *p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2938 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2939 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2940 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2941 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2942 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2943 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2944 /* current input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2945 int inpos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2946 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2947 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2948 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2949 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2950 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2951 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2952 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2953 int known_errorHandler = -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2954
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2955 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2956 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2957 return PyUnicode_EncodeLatin1(p, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2958
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2959 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2960 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2961 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2962 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2963 goto onError;
13966
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being
Marc-André Lemburg <mal@egenix.com>
parents: 13946
diff changeset
2964 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2965 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2966
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2967 while (inpos<size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2968 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2969 PyObject *x = charmapencode_output(p[inpos], mapping, &res, &respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2970 if (x==NULL) /* error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2971 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2972 if (x==Py_None) { /* unencodable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2973 if (charmap_encoding_error(p, size, &inpos, mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2974 &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2975 &known_errorHandler, errorHandler, errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2976 &res, &respos))
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
2977 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2978 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2979 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2980 /* done with this character => adjust input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2981 ++inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2982 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2983 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2984
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2985 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2986 if (respos<PyString_GET_SIZE(res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2987 if (_PyString_Resize(&res, respos))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2988 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2989 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2990 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2991 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2992 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2993
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2994 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2995 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2996 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
2997 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2998 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
2999 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3000
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3001 PyObject *PyUnicode_AsCharmapString(PyObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3002 PyObject *mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3003 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3004 if (!PyUnicode_Check(unicode) || mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3005 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3006 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3007 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3008 return PyUnicode_EncodeCharmap(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3009 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3010 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3011 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3012 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3013
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3014 /* create or adjust a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3015 static void make_translate_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3016 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3017 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3018 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3019 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3020 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3021 *exceptionObject = PyUnicodeTranslateError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3022 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3023 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3024 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3025 if (PyUnicodeTranslateError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3026 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3027 if (PyUnicodeTranslateError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3028 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3029 if (PyUnicodeTranslateError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3030 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3031 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3032 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3033 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3034 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3035 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3036 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3037
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3038 /* raises a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3039 static void raise_translate_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3040 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3041 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3042 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3043 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3044 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3045 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3046 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3047 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3048 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3049
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3050 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3051 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3052 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3053 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3054 static PyObject *unicode_translate_call_errorhandler(const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3055 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3056 const char *reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3057 const Py_UNICODE *unicode, int size, PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3058 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3059 int *newpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3060 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3061 static char *argparse = "O!i;translating error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3062
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3063 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3064 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3065
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3066 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3067 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3068 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3069 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3070 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3071
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3072 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3073 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3074 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3075 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3076
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3077 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3078 *errorHandler, *exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3079 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3080 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3081 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3082 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3083 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3084 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3085 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3086 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3087 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3088 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3089 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3090 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3091 if (*newpos<0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3092 *newpos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3093 else if (*newpos>size)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3094 *newpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3095 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3096 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3097 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3098 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3099
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3100 /* Lookup the character ch in the mapping and put the result in result,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3101 which must be decrefed by the caller.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3102 Return 0 on success, -1 on error */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3103 static
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3104 int charmaptranslate_lookup(Py_UNICODE c, PyObject *mapping, PyObject **result)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3105 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3106 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3107 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3108
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3109 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3110 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3111 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3112 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3113 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3114 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3115 /* No mapping found means: use 1:1 mapping. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3116 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3117 *result = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3118 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3119 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3120 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3121 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3122 else if (x == Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3123 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3124 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3125 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3126 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3127 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3128 long max = PyUnicode_GetMax();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3129 if (value < 0 || value > max) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3130 PyErr_Format(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3131 "character mapping must be in range(0x%lx)", max+1);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3132 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3133 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3134 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3135 *result = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3136 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3137 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3138 else if (PyUnicode_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3139 *result = x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3140 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3141 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3142 else {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3143 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3144 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3145 "character mapping must return integer, None or unicode");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3146 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3147 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3148 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3149 /* ensure that *outobj is at least requiredsize characters long,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3150 if not reallocate and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3151 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3152 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3153 int charmaptranslate_makespace(PyObject **outobj, Py_UNICODE **outp, int *outsize,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3154 int requiredsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3155 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3156 if (requiredsize > *outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3157 /* remember old output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3158 int outpos = *outp-PyUnicode_AS_UNICODE(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3159 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3160 if (requiredsize < 2 * *outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3161 requiredsize = 2 * *outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3162 if (_PyUnicode_Resize(outobj, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3163 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3164 *outp = PyUnicode_AS_UNICODE(*outobj) + outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3165 *outsize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3166 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3167 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3168 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3169 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3170 various state variables. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3171 was put in the output buffer in *result, or Py_None, if the mapping was
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3172 undefined (in which case no character was written).
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3173 The called must decref result.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3174 Return 0 on success, -1 on error. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3175 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3176 int charmaptranslate_output(Py_UNICODE c, PyObject *mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3177 PyObject **outobj, int *outsize, Py_UNICODE **outp, PyObject **res)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3178 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3179 if (charmaptranslate_lookup(c, mapping, res))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3180 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3181 if (*res==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3182 /* not found => default to 1:1 mapping */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3183 *(*outp)++ = (Py_UNICODE)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3184 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3185 else if (*res==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3186 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3187 else if (PyInt_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3188 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3189 *(*outp)++ = (Py_UNICODE)PyInt_AS_LONG(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3190 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3191 else if (PyUnicode_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3192 int repsize = PyUnicode_GET_SIZE(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3193 if (repsize==1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3194 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3195 *(*outp)++ = *PyUnicode_AS_UNICODE(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3196 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3197 else if (repsize!=0) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3198 /* more than one character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3199 int requiredsize = *outsize + repsize - 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3200 if (charmaptranslate_makespace(outobj, outp, outsize, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3201 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3202 memcpy(*outp, PyUnicode_AS_UNICODE(*res), sizeof(Py_UNICODE)*repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3203 *outp += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3204 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3205 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3206 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3207 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3208 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3209 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3210
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3211 PyObject *PyUnicode_TranslateCharmap(const Py_UNICODE *p,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3212 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3213 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3214 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3215 {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3216 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3217 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3218 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3219 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3220 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3221 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3222 Py_UNICODE *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3223 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3224 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3225 int ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3226 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3227 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3228 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3229 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3230 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3231 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3232 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3233
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3234 if (mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3235 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3236 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3237 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3238
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3239 /* allocate enough for a simple 1:1 translation without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3240 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3241 res = PyUnicode_FromUnicode(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3242 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3243 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3244 if (size == 0)
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3245 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3246 str = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3247 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3248
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3249 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3250 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3251 PyObject *x = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3252 if (charmaptranslate_output(*p, mapping, &res, &ressize, &str, &x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3253 Py_XDECREF(x);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3254 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3255 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3256 if (x!=Py_None) /* it worked => adjust input pointer */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3257 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3258 else { /* untranslatable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3259 PyObject *repunicode = NULL; /* initialize to prevent gcc warning */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3260 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3261 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3262 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3263 /* startpos for collecting untranslatable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3264 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3265 const Py_UNICODE *collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3266 const Py_UNICODE *coll;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3267
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3268 Py_XDECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3269 /* find all untranslatable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3270 while (collend < endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3271 if (charmaptranslate_lookup(*collend, mapping, &x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3272 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3273 Py_XDECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3274 if (x!=Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3275 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3276 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3277 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3278 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3279 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3280 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3281 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3282 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3283 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3284 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3285 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3286 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3287 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3288 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3289 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3290 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3291 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3292 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3293 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3294 raise_translate_exception(&exc, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3295 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3296 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3297 /* No need to check for space, this is a 1:1 replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3298 for (coll = collstart; coll<collend; ++coll)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3299 *str++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3300 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3301 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3302 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3303 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3304 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3305 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3306 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3307 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3308 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3309 sprintf(buffer, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3310 if (charmaptranslate_makespace(&res, &str, &ressize,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3311 (str-PyUnicode_AS_UNICODE(res))+strlen(buffer)+(endp-collend)))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3312 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3313 for (cp = buffer; *cp; ++cp)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3314 *str++ = *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3315 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3316 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3317 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3318 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3319 repunicode = unicode_translate_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3320 reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3321 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3322 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3323 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3324 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3325 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3326 if (charmaptranslate_makespace(&res, &str, &ressize,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3327 (str-PyUnicode_AS_UNICODE(res))+repsize+(endp-collend))) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3328 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3329 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3330 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3331 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3332 *str++ = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3333 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3334 Py_DECREF(repunicode);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3335 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3336 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3337 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3338 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3339 respos = str-PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3340 if (respos<ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3341 if (_PyUnicode_Resize(&res, respos))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3342 goto onError;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3343 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3344 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3345 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3346 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3347
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3348 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3349 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3350 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3351 Py_XDECREF(errorHandler);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3352 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3353 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3354
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3355 PyObject *PyUnicode_Translate(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3356 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3357 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3358 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3359 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3360
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3361 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3362 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3363 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3364 result = PyUnicode_TranslateCharmap(PyUnicode_AS_UNICODE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3365 PyUnicode_GET_SIZE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3366 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3367 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3368 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3369 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3370
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3371 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3372 Py_XDECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3373 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3374 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3375
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3376 /* --- Decimal Encoder ---------------------------------------------------- */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3377
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3378 int PyUnicode_EncodeDecimal(Py_UNICODE *s,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3379 int length,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3380 char *output,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3381 const char *errors)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3382 {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3383 Py_UNICODE *p, *end;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3384 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3385 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3386 const char *encoding = "decimal";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3387 const char *reason = "invalid decimal Unicode string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3388 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3389 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3390 int known_errorHandler = -1;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3391
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3392 if (output == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3393 PyErr_BadArgument();
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3394 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3395 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3396
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3397 p = s;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3398 end = s + length;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3399 while (p < end) {
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3400 register Py_UNICODE ch = *p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3401 int decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3402 PyObject *repunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3403 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3404 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3405 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3406 Py_UNICODE *collstart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3407 Py_UNICODE *collend;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3408
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3409 if (Py_UNICODE_ISSPACE(ch)) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3410 *output++ = ' ';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3411 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3412 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3413 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3414 decimal = Py_UNICODE_TODECIMAL(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3415 if (decimal >= 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3416 *output++ = '0' + decimal;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3417 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3418 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3419 }
12275
cbfa95a6a03e Conrad Huang points out that "if (0 < ch < 256)", while legal C,
Guido van Rossum <guido@python.org>
parents: 12249
diff changeset
3420 if (0 < ch && ch < 256) {
12577
85f00f8cd674 Fix warning detected by VC++ on assignment of Py_UNICODE to char.
Guido van Rossum <guido@python.org>
parents: 12576
diff changeset
3421 *output++ = (char)ch;
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3422 ++p;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3423 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3424 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3425 /* All other characters are considered unencodable */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3426 collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3427 collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3428 while (collend < end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3429 if ((0 < *collend && *collend < 256) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3430 !Py_UNICODE_ISSPACE(*collend) ||
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3431 Py_UNICODE_TODECIMAL(*collend))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3432 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3433 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3434 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3435 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3436 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3437 if ((errors==NULL) || (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3438 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3439 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3440 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3441 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3442 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3443 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3444 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3445 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3446 known_errorHandler = 0;
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3447 }
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3448 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3449 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3450 raise_encode_exception(&exc, encoding, s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3451 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3452 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3453 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3454 *output++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3455 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3456 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3457 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3458 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3459 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3460 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3461 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3462 output += sprintf(output, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3463 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3464 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3465 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3466 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3467 encoding, reason, s, length, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3468 collstart-s, collend-s, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3469 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3470 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3471 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3472 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3473 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3474 Py_UNICODE ch = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3475 if (Py_UNICODE_ISSPACE(ch))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3476 *output++ = ' ';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3477 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3478 decimal = Py_UNICODE_TODECIMAL(ch);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3479 if (decimal >= 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3480 *output++ = '0' + decimal;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3481 else if (0 < ch && ch < 256)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3482 *output++ = (char)ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3483 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3484 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3485 raise_encode_exception(&exc, encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3486 s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3487 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3488 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3489 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3490 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3491 p = s + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3492 Py_DECREF(repunicode);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3493 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3494 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3495 /* 0-terminate the output string */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3496 *output++ = '\0';
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3497 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3498 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3499 return 0;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3500
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3501 onError:
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3502 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
3503 Py_XDECREF(errorHandler);
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3504 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3505 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
3506
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3507 /* --- Helpers ------------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3508
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3509 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3510 int count(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3511 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3512 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3513 PyUnicodeObject *substring)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3514 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3515 int count = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3516
15910
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3517 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3518 start += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3519 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3520 start = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3521 if (end > self->length)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3522 end = self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3523 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3524 end += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3525 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3526 end = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core
Marc-André Lemburg <mal@egenix.com>
parents: 15749
diff changeset
3527
12957
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3528 if (substring->length == 0)
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3529 return (end - start + 1);
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3530
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3531 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3532
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3533 while (start <= end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3534 if (Py_UNICODE_MATCH(self, start, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3535 count++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3536 start += substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3537 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3538 start++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3539
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3540 return count;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3541 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3542
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3543 int PyUnicode_Count(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3544 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3545 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3546 int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3547 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3548 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3549
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3550 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3551 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3552 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3553 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3554 if (substr == NULL) {
12957
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12939
diff changeset
3555 Py_DECREF(str);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3556 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3557 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3558
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3559 result = count((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3560 start, end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3561 (PyUnicodeObject *)substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3562
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3563 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3564 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3565 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3566 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3567
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3568 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3569 int findstring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3570 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3571 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3572 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3573 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3574 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3575 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3576 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3577 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3578 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3579
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3580 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3581 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3582 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3583 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3584 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3585 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3586
24918
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith().
Guido van Rossum <guido@python.org>
parents: 24831
diff changeset
3587 if (substring->length == 0)
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith().
Guido van Rossum <guido@python.org>
parents: 24831
diff changeset
3588 return (direction > 0) ? start : end;
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith().
Guido van Rossum <guido@python.org>
parents: 24831
diff changeset
3589
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3590 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3591
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3592 if (direction < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3593 for (; end >= start; end--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3594 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3595 return end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3596 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3597 for (; start <= end; start++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3598 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3599 return start;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3600 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3601
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3602 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3603 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3604
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3605 int PyUnicode_Find(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3606 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3607 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3608 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3609 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3610 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3611 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3613 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3614 if (str == NULL)
23561
c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents: 23542
diff changeset
3615 return -2;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3616 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3617 if (substr == NULL) {
23562
eaca627f2a54 Fix a possible segfault. Found be Neal Norvitz.
Marc-André Lemburg <mal@egenix.com>
parents: 23561
diff changeset
3618 Py_DECREF(str);
23561
c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error
Marc-André Lemburg <mal@egenix.com>
parents: 23542
diff changeset
3619 return -2;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3620 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3621
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3622 result = findstring((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3623 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3624 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3625 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3626 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3627 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3628 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3629
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3630 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3631 int tailmatch(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3632 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3633 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3634 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3635 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3636 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3637 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3638 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3639 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3640 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3641
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3642 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3643 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3644
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3645 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3646 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3647 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3648 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3649 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3650 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3651
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3652 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3653 if (end < start)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3654 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3655
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3656 if (direction > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3657 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3658 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3659 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3660 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3661 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3662 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3663
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3664 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3665 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3666
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3667 int PyUnicode_Tailmatch(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3668 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3669 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3670 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3671 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3672 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3673 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3674
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3675 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3676 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3677 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3678 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3679 if (substr == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3680 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3681 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3682 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3683
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3684 result = tailmatch((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3685 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3686 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3687 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3688 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3689 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3690 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3691
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3692 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3693 const Py_UNICODE *findchar(const Py_UNICODE *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3694 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3695 Py_UNICODE ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3696 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3697 /* like wcschr, but doesn't stop at NULL characters */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3698
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3699 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3700 if (*s == ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3701 return s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3702 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3703 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3704
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3705 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3706 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3707
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3708 /* Apply fixfct filter to the Unicode object self and return a
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3709 reference to the modified object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3710
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3711 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3712 PyObject *fixup(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3713 int (*fixfct)(PyUnicodeObject *s))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3714 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3715
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3716 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3717
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3718 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3719 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3720 return NULL;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3721
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3722 Py_UNICODE_COPY(u->str, self->str, self->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3723
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
3724 if (!fixfct(u) && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3725 /* fixfct should return TRUE if it modified the buffer. If
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3726 FALSE, return a reference to the original buffer instead
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3727 (to save space, not time) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3728 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3729 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3730 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3731 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3732 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3733 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3734
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3735 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3736 int fixupper(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3737 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3738 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3739 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3740 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3741
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3742 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3743 register Py_UNICODE ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3744
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3745 ch = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3746 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3747 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3748 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3749 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3750 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3751 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3752
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3753 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3754 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3755
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3756 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3757 int fixlower(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3758 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3759 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3760 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3761 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3762
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3763 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3764 register Py_UNICODE ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3765
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3766 ch = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3767 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3768 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3769 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3770 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3771 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3772 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3773
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3774 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3775 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3776
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3777 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3778 int fixswapcase(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3779 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3780 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3781 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3782 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3783
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3784 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3785 if (Py_UNICODE_ISUPPER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3786 *s = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3787 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3788 } else if (Py_UNICODE_ISLOWER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3789 *s = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3790 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3791 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3792 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3793 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3794
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3795 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3796 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3797
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3798 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3799 int fixcapitalize(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3800 {
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3801 int len = self->length;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3802 Py_UNICODE *s = self->str;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3803 int status = 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3804
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3805 if (len == 0)
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3806 return 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3807 if (Py_UNICODE_ISLOWER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3808 *s = Py_UNICODE_TOUPPER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3809 status = 1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3810 }
16332
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3811 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3812 while (--len > 0) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3813 if (Py_UNICODE_ISUPPER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3814 *s = Py_UNICODE_TOLOWER(*s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3815 status = 1;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3816 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3817 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3818 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the
Marc-André Lemburg <mal@egenix.com>
parents: 16226
diff changeset
3819 return status;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3820 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3821
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3822 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3823 int fixtitle(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3824 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3825 register Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3826 register Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3827 int previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3828
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3829 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3830 if (PyUnicode_GET_SIZE(self) == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3831 Py_UNICODE ch = Py_UNICODE_TOTITLE(*p);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3832 if (*p != ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3833 *p = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3834 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3835 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3836 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3837 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3838 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3839
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3840 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3841 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3842 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3843 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3844
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3845 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3846 *p = Py_UNICODE_TOLOWER(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3847 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3848 *p = Py_UNICODE_TOTITLE(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3849
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3850 if (Py_UNICODE_ISLOWER(ch) ||
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3851 Py_UNICODE_ISUPPER(ch) ||
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3852 Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3853 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3854 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3855 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3856 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3857 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3858 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3859
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3860 PyObject *PyUnicode_Join(PyObject *separator,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3861 PyObject *seq)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3862 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3863 Py_UNICODE *sep;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3864 int seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3865 PyUnicodeObject *res = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3866 int reslen = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3867 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3868 int sz = 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3869 int i;
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3870 PyObject *it;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3871
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3872 it = PyObject_GetIter(seq);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3873 if (it == NULL)
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3874 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3875
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3876 if (separator == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3877 Py_UNICODE blank = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3878 sep = &blank;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3879 seplen = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3880 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3881 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3882 separator = PyUnicode_FromObject(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3883 if (separator == NULL)
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3884 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3885 sep = PyUnicode_AS_UNICODE(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3886 seplen = PyUnicode_GET_SIZE(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3887 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3888
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3889 res = _PyUnicode_New(sz);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3890 if (res == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3891 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3892 p = PyUnicode_AS_UNICODE(res);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3893 reslen = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3894
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3895 for (i = 0; ; ++i) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3896 int itemlen;
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3897 PyObject *item = PyIter_Next(it);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3898 if (item == NULL) {
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3899 if (PyErr_Occurred())
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3900 goto onError;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3901 break;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3902 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3903 if (!PyUnicode_Check(item)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3904 PyObject *v;
20007
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3905 if (!PyString_Check(item)) {
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3906 PyErr_Format(PyExc_TypeError,
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3907 "sequence item %i: expected string or Unicode,"
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3908 " %.80s found",
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3909 i, item->ob_type->tp_name);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3910 Py_DECREF(item);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3911 goto onError;
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3912 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3913 v = PyUnicode_FromObject(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3914 Py_DECREF(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3915 item = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3916 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3917 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3918 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3919 itemlen = PyUnicode_GET_SIZE(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3920 while (reslen + itemlen + seplen >= sz) {
20007
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3921 if (_PyUnicode_Resize(&res, sz*2)) {
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3922 Py_DECREF(item);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3923 goto onError;
20007
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence
Marc-André Lemburg <mal@egenix.com>
parents: 19999
diff changeset
3924 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3925 sz *= 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3926 p = PyUnicode_AS_UNICODE(res) + reslen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3927 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3928 if (i > 0) {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3929 Py_UNICODE_COPY(p, sep, seplen);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3930 p += seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3931 reslen += seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3932 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3933 Py_UNICODE_COPY(p, PyUnicode_AS_UNICODE(item), itemlen);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3934 p += itemlen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3935 reslen += itemlen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3936 Py_DECREF(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3937 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
3938 if (_PyUnicode_Resize(&res, reslen))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3939 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3940
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3941 Py_XDECREF(separator);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3942 Py_DECREF(it);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3943 return (PyObject *)res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3944
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3945 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3946 Py_XDECREF(separator);
17687
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3947 Py_XDECREF(res);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change
Tim Peters <tim.peters@gmail.com>
parents: 17640
diff changeset
3948 Py_DECREF(it);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3949 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3950 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3951
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3952 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3953 PyUnicodeObject *pad(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3954 int left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3955 int right,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3956 Py_UNICODE fill)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3957 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3958 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3959
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3960 if (left < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3961 left = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3962 if (right < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3963 right = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3964
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
3965 if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3966 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3967 return self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3968 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3969
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3970 u = _PyUnicode_New(left + self->length + right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3971 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3972 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3973 Py_UNICODE_FILL(u->str, fill, left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3974 Py_UNICODE_COPY(u->str + left, self->str, self->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3975 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3976 Py_UNICODE_FILL(u->str + left + self->length, fill, right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3977 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3978
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3979 return u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3980 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3981
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3982 #define SPLIT_APPEND(data, left, right) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3983 str = PyUnicode_FromUnicode(data + left, right - left); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3984 if (!str) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3985 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3986 if (PyList_Append(list, str)) { \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3987 Py_DECREF(str); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3988 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3989 } \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3990 else \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3991 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3992
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3993 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3994 PyObject *split_whitespace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3995 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3996 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3997 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3998 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
3999 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4000 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4001 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4002
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4003 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4004 /* find a token */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4005 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4006 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4007 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4008 while (i < len && !Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4009 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4010 if (j < i) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4011 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4012 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4013 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4014 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4015 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4016 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4017 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4018 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4019 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4020 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4021 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4022 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4023
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4024 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4025 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4026 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4027 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4028
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4029 PyObject *PyUnicode_Splitlines(PyObject *string,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4030 int keepends)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4031 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4032 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4033 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4034 int len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4035 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4036 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4037 Py_UNICODE *data;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4038
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4039 string = PyUnicode_FromObject(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4040 if (string == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4041 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4042 data = PyUnicode_AS_UNICODE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4043 len = PyUnicode_GET_SIZE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4044
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4045 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4046 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4047 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4048
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4049 for (i = j = 0; i < len; ) {
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4050 int eol;
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4051
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4052 /* Find a line and append it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4053 while (i < len && !Py_UNICODE_ISLINEBREAK(data[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4054 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4055
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4056 /* Skip the line break reading CRLF as one line break */
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4057 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4058 if (i < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4059 if (data[i] == '\r' && i + 1 < len &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4060 data[i+1] == '\n')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4061 i += 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4062 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4063 i++;
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4064 if (keepends)
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4065 eol = i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4066 }
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
4067 SPLIT_APPEND(data, j, eol);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4068 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4069 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4070 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4071 SPLIT_APPEND(data, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4072 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4073
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4074 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4075 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4076
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4077 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4078 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4079 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4080 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4081 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4082
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4083 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4084 PyObject *split_char(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4085 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4086 Py_UNICODE ch,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4087 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4088 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4089 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4090 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4091 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4092 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4093
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4094 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4095 if (self->str[i] == ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4096 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4097 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4098 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4099 i = j = i + 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4100 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4101 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4102 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4103 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4104 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4105 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4106 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4107
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4108 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4109 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4110 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4111 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4112
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4113 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4114 PyObject *split_substring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4115 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4116 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4117 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4118 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4119 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4120 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4121 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4122 int sublen = substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4123 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4124
15609
0906c8e2c961 Fix off-by-one error in split_substring(). Fixes SF bug #122162.
Guido van Rossum <guido@python.org>
parents: 15579
diff changeset
4125 for (i = j = 0; i <= len - sublen; ) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4126 if (Py_UNICODE_MATCH(self, i, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4127 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4128 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4129 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4130 i = j = i + sublen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4131 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4132 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4133 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4134 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4135 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4136 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4137 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4138
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4139 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4140 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4141 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4142 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4143
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4144 #undef SPLIT_APPEND
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4145
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4146 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4147 PyObject *split(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4148 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4149 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4150 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4151 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4152
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4153 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4154 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4155
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4156 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4157 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4158 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4159
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4160 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4161 return split_whitespace(self,list,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4162
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4163 else if (substring->length == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4164 return split_char(self,list,substring->str[0],maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4165
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4166 else if (substring->length == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4167 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4168 PyErr_SetString(PyExc_ValueError, "empty separator");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4169 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4170 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4171 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4172 return split_substring(self,list,substring,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4173 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4174
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4175 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4176 PyObject *replace(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4177 PyUnicodeObject *str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4178 PyUnicodeObject *str2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4179 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4180 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4181 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4182
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4183 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4184 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4185
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4186 if (str1->length == 1 && str2->length == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4187 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4188
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4189 /* replace characters */
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4190 if (!findchar(self->str, self->length, str1->str[0]) &&
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4191 PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4192 /* nothing to replace, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4193 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4194 u = self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4195 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4196 Py_UNICODE u1 = str1->str[0];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4197 Py_UNICODE u2 = str2->str[0];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4198
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4199 u = (PyUnicodeObject*) PyUnicode_FromUnicode(
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4200 NULL,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4201 self->length
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4202 );
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4203 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4204 Py_UNICODE_COPY(u->str, self->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4205 self->length);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4206 for (i = 0; i < u->length; i++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4207 if (u->str[i] == u1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4208 if (--maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4209 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4210 u->str[i] = u2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4211 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4212 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
4213 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4214
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4215 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4216 int n, i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4217 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4218
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4219 /* replace strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4220 n = count(self, 0, self->length, str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4221 if (n > maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4222 n = maxcount;
25005
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4223 if (n == 0) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4224 /* nothing to replace, return original string */
25005
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4225 if (PyUnicode_CheckExact(self)) {
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4226 Py_INCREF(self);
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4227 u = self;
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4228 }
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4229 else {
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4230 u = (PyUnicodeObject *)
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4231 PyUnicode_FromUnicode(self->str, self->length);
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the
Guido van Rossum <guido@python.org>
parents: 25004
diff changeset
4232 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4233 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4234 u = _PyUnicode_New(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4235 self->length + n * (str2->length - str1->length));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4236 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4237 i = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4238 p = u->str;
25004
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4239 if (str1->length > 0) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4240 while (i <= self->length - str1->length)
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4241 if (Py_UNICODE_MATCH(self, i, str1)) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4242 /* replace string segment */
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4243 Py_UNICODE_COPY(p, str2->str, str2->length);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4244 p += str2->length;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4245 i += str1->length;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4246 if (--n <= 0) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4247 /* copy remaining part */
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4248 Py_UNICODE_COPY(p, self->str+i, self->length-i);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4249 break;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4250 }
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4251 } else
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4252 *p++ = self->str[i++];
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4253 } else {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4254 while (n > 0) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4255 Py_UNICODE_COPY(p, str2->str, str2->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4256 p += str2->length;
25004
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4257 if (--n <= 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4258 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4259 *p++ = self->str[i++];
25004
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4260 }
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4261 Py_UNICODE_COPY(p, self->str+i, self->length-i);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement
Guido van Rossum <guido@python.org>
parents: 24918
diff changeset
4262 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4264 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4265 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4266
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4267 return (PyObject *) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4268 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4269
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4270 /* --- Unicode Object Methods --------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4271
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4272 PyDoc_STRVAR(title__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4273 "S.title() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4274 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4275 Return a titlecased version of S, i.e. words start with title case\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4276 characters, all remaining cased characters have lower case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4277
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4278 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4279 unicode_title(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4280 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4281 return fixup(self, fixtitle);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4282 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4283
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4284 PyDoc_STRVAR(capitalize__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4285 "S.capitalize() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4286 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4287 Return a capitalized version of S, i.e. make the first character\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4288 have upper case.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4289
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4290 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4291 unicode_capitalize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4292 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4293 return fixup(self, fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4294 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4295
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4296 #if 0
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4297 PyDoc_STRVAR(capwords__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4298 "S.capwords() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4299 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4300 Apply .capitalize() to all words in S and return the result with\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4301 normalized whitespace (all whitespace strings are replaced by ' ').");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4302
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4303 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4304 unicode_capwords(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4305 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4306 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4307 PyObject *item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4308 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4309
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4310 /* Split into words */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4311 list = split(self, NULL, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4312 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4313 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4314
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4315 /* Capitalize each word */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4316 for (i = 0; i < PyList_GET_SIZE(list); i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4317 item = fixup((PyUnicodeObject *)PyList_GET_ITEM(list, i),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4318 fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4319 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4320 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4321 Py_DECREF(PyList_GET_ITEM(list, i));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4322 PyList_SET_ITEM(list, i, item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4323 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4324
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4325 /* Join the words to form a new string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4326 item = PyUnicode_Join(NULL, list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4327
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4328 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4329 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4330 return (PyObject *)item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4331 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4332 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4333
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4334 PyDoc_STRVAR(center__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4335 "S.center(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4336 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4337 Return S centered in a Unicode string of length width. Padding is done\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4338 using spaces.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4340 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4341 unicode_center(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4342 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4343 int marg, left;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4344 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4345
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4346 if (!PyArg_ParseTuple(args, "i:center", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4347 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4348
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
4349 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4350 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4351 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4352 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4353
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4354 marg = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4355 left = marg / 2 + (marg & width & 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4356
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4357 return (PyObject*) pad(self, left, marg - left, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4358 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4359
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4360 #if 0
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4361
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4362 /* This code should go into some future Unicode collation support
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4363 module. The basic comparison should compare ordinals on a naive
13946
7a99450da2d6 Clean up warning from Monterey compiler.
Trent Mick <trentm@activestate.com>
parents: 13890
diff changeset
4364 basis (this is what Java does and thus JPython too). */
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4365
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4366 /* speedy UTF-16 code point order comparison */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4367 /* gleaned from: */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4368 /* http://www-4.ibm.com/software/developer/library/utf16.html?dwzone=unicode */
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4369
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
4370 static short utf16Fixup[32] =
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4371 {
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4372 0, 0, 0, 0, 0, 0, 0, 0,
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4373 0, 0, 0, 0, 0, 0, 0, 0,
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4374 0, 0, 0, 0, 0, 0, 0, 0,
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
4375 0, 0, 0, 0x2000, -0x800, -0x800, -0x800, -0x800
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4376 };
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4377
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4378 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4379 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4380 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4381 int len1, len2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4382
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4383 Py_UNICODE *s1 = str1->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4384 Py_UNICODE *s2 = str2->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4385
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4386 len1 = str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4387 len2 = str2->length;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4388
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4389 while (len1 > 0 && len2 > 0) {
13411
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt.
Marc-André Lemburg <mal@egenix.com>
parents: 13401
diff changeset
4390 Py_UNICODE c1, c2;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4391
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4392 c1 = *s1++;
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4393 c2 = *s2++;
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4394
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4395 if (c1 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4396 c1 += utf16Fixup[c1>>11];
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4397 if (c2 > (1<<11) * 26)
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4398 c2 += utf16Fixup[c2>>11];
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4399 /* now c1 and c2 are in UTF-32-compatible order */
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4400
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4401 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4402 return (c1 < c2) ? -1 : 1;
13337
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4403
e101600adee4 Bill Tutt:
Marc-André Lemburg <mal@egenix.com>
parents: 13209
diff changeset
4404 len1--; len2--;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4405 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4406
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4407 return (len1 < len2) ? -1 : (len1 != len2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4408 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4409
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4410 #else
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4411
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4412 static int
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4413 unicode_compare(PyUnicodeObject *str1, PyUnicodeObject *str2)
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4414 {
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4415 register int len1, len2;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4416
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4417 Py_UNICODE *s1 = str1->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4418 Py_UNICODE *s2 = str2->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4419
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4420 len1 = str1->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4421 len2 = str2->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4422
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4423 while (len1 > 0 && len2 > 0) {
18279
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4424 Py_UNICODE c1, c2;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4425
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4426 c1 = *s1++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4427 c2 = *s2++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4428
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4429 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4430 return (c1 < c2) ? -1 : 1;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case
Fredrik Lundh <fredrik@pythonware.com>
parents: 18278
diff changeset
4431
13890
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4432 len1--; len2--;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4433 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4434
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4435 return (len1 < len2) ? -1 : (len1 != len2);
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4436 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4437
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4438 #endif
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare
Marc-André Lemburg <mal@egenix.com>
parents: 13837
diff changeset
4439
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4440 int PyUnicode_Compare(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4441 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4442 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4443 PyUnicodeObject *u = NULL, *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4444 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4445
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4446 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4447 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4448 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4449 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4450 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4451 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4452 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4453
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13576
diff changeset
4454 /* Shortcut for empty or interned objects */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4455 if (v == u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4456 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4457 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4458 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4459 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4460
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4461 result = unicode_compare(u, v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4462
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4463 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4464 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4465 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4466
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4467 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4468 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4469 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4470 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4471 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4472
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4473 int PyUnicode_Contains(PyObject *container,
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4474 PyObject *element)
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4475 {
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4476 PyUnicodeObject *u = NULL, *v = NULL;
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4477 int result, size;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4478 register const Py_UNICODE *lhs, *end, *rhs;
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4479
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4480 /* Coerce the two arguments */
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4481 v = (PyUnicodeObject *)PyUnicode_FromObject(element);
13049
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12957
diff changeset
4482 if (v == NULL) {
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12957
diff changeset
4483 PyErr_SetString(PyExc_TypeError,
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4484 "'in <string>' requires string as left operand");
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4485 goto onError;
13049
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12957
diff changeset
4486 }
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4487 u = (PyUnicodeObject *)PyUnicode_FromObject(container);
25895
509bb0b6dfa3 Fix for bug #626172: crash using unicode latin1 single char
Marc-André Lemburg <mal@egenix.com>
parents: 25719
diff changeset
4488 if (u == NULL)
12248
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that
Guido van Rossum <guido@python.org>
parents: 12193
diff changeset
4489 goto onError;
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4490
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4491 size = PyUnicode_GET_SIZE(v);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4492 rhs = PyUnicode_AS_UNICODE(v);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4493 lhs = PyUnicode_AS_UNICODE(u);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4494
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4495 result = 0;
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4496 if (size == 1) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4497 end = lhs + PyUnicode_GET_SIZE(u);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4498 while (lhs < end) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4499 if (*lhs++ == *rhs) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4500 result = 1;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4501 break;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4502 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4503 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4504 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4505 else {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4506 end = lhs + (PyUnicode_GET_SIZE(u) - size);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4507 while (lhs <= end) {
24608
b6707e159909 PyUnicode_Contains(): The memcmp() call didn't take into account the
Barry Warsaw <barry@python.org>
parents: 24598
diff changeset
4508 if (memcmp(lhs++, rhs, size * sizeof(Py_UNICODE)) == 0) {
24598
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4509 result = 1;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4510 break;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a
Barry Warsaw <barry@python.org>
parents: 24392
diff changeset
4511 }
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4512 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4513 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4514
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4515 Py_DECREF(u);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4516 Py_DECREF(v);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4517 return result;
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4518
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4519 onError:
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4520 Py_XDECREF(u);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4521 Py_XDECREF(v);
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4522 return -1;
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4523 }
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
4524
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4525 /* Concat to string or Unicode object giving a new Unicode object. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4526
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4527 PyObject *PyUnicode_Concat(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4528 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4529 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4530 PyUnicodeObject *u = NULL, *v = NULL, *w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4531
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4532 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4533 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4534 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4535 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4536 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4537 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4538 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4539
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4540 /* Shortcuts */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4541 if (v == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4542 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4543 return (PyObject *)u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4544 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4545 if (u == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4546 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4547 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4548 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4549
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4550 /* Concat the two Unicode strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4551 w = _PyUnicode_New(u->length + v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4552 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4553 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4554 Py_UNICODE_COPY(w->str, u->str, u->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4555 Py_UNICODE_COPY(w->str + u->length, v->str, v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4556
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4557 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4558 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4559 return (PyObject *)w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4560
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4561 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4562 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4563 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4564 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4565 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4566
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4567 PyDoc_STRVAR(count__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4568 "S.count(sub[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4569 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4570 Return the number of occurrences of substring sub in Unicode string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4571 S[start:end]. Optional arguments start and end are\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4572 interpreted as in slice notation.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4573
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4574 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4575 unicode_count(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4576 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4577 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4578 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4579 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4580 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4581
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
4582 if (!PyArg_ParseTuple(args, "O|O&O&:count", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
4583 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4584 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4585
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4586 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4587 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4588 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4589 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4590
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4591 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4592 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4593 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4594 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4595 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4596 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4597 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4598 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4599 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4600 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4601
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4602 result = PyInt_FromLong((long) count(self, start, end, substring));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4603
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4604 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4605 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4606 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4607
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4608 PyDoc_STRVAR(encode__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4609 "S.encode([encoding[,errors]]) -> string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4610 \n\
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
4611 Return an encoded string version of S. Default encoding is the current\n\
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
4612 default string encoding. errors may be given to set a different error\n\
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
4613 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
25091
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4614 a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4615 'xmlcharrefreplace' as well as any other name registered with\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401)
Walter Dörwald <walter@livinglogic.de>
parents: 25005
diff changeset
4616 codecs.register_error that can handle UnicodeEncodeErrors.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4617
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4618 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4619 unicode_encode(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4620 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4621 char *encoding = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4622 char *errors = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4623 if (!PyArg_ParseTuple(args, "|ss:encode", &encoding, &errors))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4624 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4625 return PyUnicode_AsEncodedString((PyObject *)self, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4626 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4627
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4628 PyDoc_STRVAR(expandtabs__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4629 "S.expandtabs([tabsize]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4630 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4631 Return a copy of S where all tab characters are expanded using spaces.\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4632 If tabsize is not given, a tab size of 8 characters is assumed.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4633
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4634 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4635 unicode_expandtabs(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4636 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4637 Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4638 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4639 Py_UNICODE *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4640 int i, j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4641 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4642 int tabsize = 8;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4643
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4644 if (!PyArg_ParseTuple(args, "|i:expandtabs", &tabsize))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4645 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4646
13620
5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either
Thomas Wouters <thomas@python.org>
parents: 13576
diff changeset
4647 /* First pass: determine size of output string */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4648 i = j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4649 e = self->str + self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4650 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4651 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4652 if (tabsize > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4653 j += tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4654 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4655 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4656 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4657 if (*p == '\n' || *p == '\r') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4658 i += j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4659 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4660 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4661 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4662
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4663 /* Second pass: create output string and fill it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4664 u = _PyUnicode_New(i + j);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4665 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4666 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4667
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4668 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4669 q = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4670
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4671 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4672 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4673 if (tabsize > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4674 i = tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4675 j += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4676 while (i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4677 *q++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4678 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4679 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4680 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4681 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4682 *q++ = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4683 if (*p == '\n' || *p == '\r')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4684 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4685 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4686
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4687 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4688 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4689
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4690 PyDoc_STRVAR(find__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4691 "S.find(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4692 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4693 Return the lowest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4694 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4695 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4696 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4697 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4698
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4699 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4700 unicode_find(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4701 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4702 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4703 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4704 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4705 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4706
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
4707 if (!PyArg_ParseTuple(args, "O|O&O&:find", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
4708 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4709 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4710 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4711 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4712 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4713 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4714
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4715 result = PyInt_FromLong(findstring(self, substring, start, end, 1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4716
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4717 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4718 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4719 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4720
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4721 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4722 unicode_getitem(PyUnicodeObject *self, int index)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4723 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4724 if (index < 0 || index >= self->length) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4725 PyErr_SetString(PyExc_IndexError, "string index out of range");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4726 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4727 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4728
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4729 return (PyObject*) PyUnicode_FromUnicode(&self->str[index], 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4730 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4731
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4732 static long
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4733 unicode_hash(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4734 {
13522
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4735 /* Since Unicode objects compare equal to their ASCII string
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4736 counterparts, they should use the individual character values
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4737 as basis for their hash value. This is needed to assure that
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4738 strings and Unicode objects behave in the same way as
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4739 dictionary keys. */
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4740
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4741 register int len;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4742 register Py_UNICODE *p;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4743 register long x;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4744
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4745 if (self->hash != -1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4746 return self->hash;
13522
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4747 len = PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4748 p = PyUnicode_AS_UNICODE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4749 x = *p << 7;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4750 while (--len >= 0)
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4751 x = (1000003*x) ^ *p++;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4752 x ^= PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4753 if (x == -1)
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4754 x = -2;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4755 self->hash = x;
acc4638f37d5 - changed hash calculation for unicode strings. the new
Fredrik Lundh <fredrik@pythonware.com>
parents: 13411
diff changeset
4756 return x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4757 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4758
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4759 PyDoc_STRVAR(index__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4760 "S.index(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4761 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4762 Like S.find() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4763
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4764 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4765 unicode_index(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4766 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4767 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4768 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4769 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4770 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4771
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
4772 if (!PyArg_ParseTuple(args, "O|O&O&:index", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
4773 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4774 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4775
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4776 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4777 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4778 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4779 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4780
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4781 result = findstring(self, substring, start, end, 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4782
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4783 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4784 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4785 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4786 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4787 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4788 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4789 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4790
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4791 PyDoc_STRVAR(islower__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4792 "S.islower() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4793 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4794 Return True if all cased characters in S are lowercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4795 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4796
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4797 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4798 unicode_islower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4799 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4800 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4801 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4802 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4803
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4804 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4805 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4806 return PyBool_FromLong(Py_UNICODE_ISLOWER(*p));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4807
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4808 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4809 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4810 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4811
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4812 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4813 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4814 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4815 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4816
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4817 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4818 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4819 else if (!cased && Py_UNICODE_ISLOWER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4820 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4821 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4822 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4823 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4824
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4825 PyDoc_STRVAR(isupper__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4826 "S.isupper() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4827 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4828 Return True if all cased characters in S are uppercase and there is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4829 at least one cased character in S, False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4830
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4831 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4832 unicode_isupper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4833 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4834 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4835 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4836 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4837
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4838 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4839 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4840 return PyBool_FromLong(Py_UNICODE_ISUPPER(*p) != 0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4841
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4842 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4843 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4844 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4845
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4846 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4847 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4848 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4849 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4850
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4851 if (Py_UNICODE_ISLOWER(ch) || Py_UNICODE_ISTITLE(ch))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4852 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4853 else if (!cased && Py_UNICODE_ISUPPER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4854 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4855 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4856 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4857 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4858
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4859 PyDoc_STRVAR(istitle__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4860 "S.istitle() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4861 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4862 Return True if S is a titlecased string, i.e. upper- and titlecase\n\
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4863 characters may only follow uncased characters and lowercase characters\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4864 only cased ones. Return False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4865
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4866 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4867 unicode_istitle(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4868 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4869 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4870 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4871 int cased, previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4872
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4873 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4874 if (PyUnicode_GET_SIZE(self) == 1)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4875 return PyBool_FromLong((Py_UNICODE_ISTITLE(*p) != 0) ||
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4876 (Py_UNICODE_ISUPPER(*p) != 0));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4877
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4878 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4879 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4880 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4881
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4882 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4883 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4884 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4885 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4886 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4887
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4888 if (Py_UNICODE_ISUPPER(ch) || Py_UNICODE_ISTITLE(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4889 if (previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4890 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4891 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4892 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4893 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4894 else if (Py_UNICODE_ISLOWER(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4895 if (!previous_is_cased)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4896 return PyBool_FromLong(0);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4897 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4898 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4899 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4900 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4901 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4902 }
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4903 return PyBool_FromLong(cased);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4904 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4905
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4906 PyDoc_STRVAR(isspace__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4907 "S.isspace() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4908 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4909 Return True if there are only whitespace characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4910 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4911
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4912 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4913 unicode_isspace(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4914 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4915 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4916 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4917
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4918 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4919 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4920 Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4921 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4922
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4923 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4924 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4925 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
4926
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4927 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4928 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4929 if (!Py_UNICODE_ISSPACE(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4930 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4931 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4932 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4933 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4934
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4935 PyDoc_STRVAR(isalpha__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4936 "S.isalpha() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4937 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4938 Return True if all characters in S are alphabetic\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4939 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4940
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4941 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4942 unicode_isalpha(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4943 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4944 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4945 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4946
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4947 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4948 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4949 Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4950 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4951
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4952 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4953 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4954 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4955
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4956 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4957 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4958 if (!Py_UNICODE_ISALPHA(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4959 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4960 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4961 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4962 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4963
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4964 PyDoc_STRVAR(isalnum__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4965 "S.isalnum() -> bool\n\
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4966 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4967 Return True if all characters in S are alphanumeric\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4968 and there is at least one character in S, False otherwise.");
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4969
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4970 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
4971 unicode_isalnum(PyUnicodeObject *self)
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4972 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4973 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4974 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4975
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4976 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4977 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4978 Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4979 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4980
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4981 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4982 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4983 return PyBool_FromLong(0);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4984
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4985 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4986 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4987 if (!Py_UNICODE_ISALNUM(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4988 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4989 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4990 return PyBool_FromLong(1);
13356
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4991 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces
Marc-André Lemburg <mal@egenix.com>
parents: 13337
diff changeset
4992
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4993 PyDoc_STRVAR(isdecimal__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4994 "S.isdecimal() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4995 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
4996 Return True if there are only decimal characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
4997 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4998
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
4999 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5000 unicode_isdecimal(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5001 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5002 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5003 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5004
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5005 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5006 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5007 Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5008 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5009
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5010 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5011 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5012 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5013
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5014 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5015 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5016 if (!Py_UNICODE_ISDECIMAL(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5017 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5018 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5019 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5020 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5021
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5022 PyDoc_STRVAR(isdigit__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5023 "S.isdigit() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5024 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5025 Return True if there are only digit characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5026 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5027
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5028 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5029 unicode_isdigit(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5030 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5031 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5032 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5033
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5034 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5035 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5036 Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5037 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5038
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5039 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5040 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5041 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5042
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5043 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5044 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5045 if (!Py_UNICODE_ISDIGIT(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5046 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5047 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5048 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5049 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5050
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5051 PyDoc_STRVAR(isnumeric__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5052 "S.isnumeric() -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5053 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5054 Return True if there are only numeric characters in S,\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5055 False otherwise.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5056
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5057 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5058 unicode_isnumeric(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5059 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5060 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5061 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5062
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5063 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5064 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5065 Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5066 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5067
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5068 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5069 if (PyString_GET_SIZE(self) == 0)
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5070 return PyBool_FromLong(0);
12924
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12909
diff changeset
5071
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5072 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5073 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5074 if (!Py_UNICODE_ISNUMERIC(*p))
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5075 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5076 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5077 return PyBool_FromLong(1);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5078 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5079
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5080 PyDoc_STRVAR(join__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5081 "S.join(sequence) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5082 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5083 Return a string which is the concatenation of the strings in the\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5084 sequence. The separator between elements is S.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5085
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5086 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5087 unicode_join(PyObject *self, PyObject *data)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5088 {
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5089 return PyUnicode_Join(self, data);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5090 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5091
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5092 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5093 unicode_length(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5094 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5095 return self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5096 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5097
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5098 PyDoc_STRVAR(ljust__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5099 "S.ljust(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5100 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5101 Return S left justified in a Unicode string of length width. Padding is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5102 done using spaces.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5103
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5104 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5105 unicode_ljust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5106 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5107 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5108 if (!PyArg_ParseTuple(args, "i:ljust", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5109 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5110
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5111 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5112 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5113 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5114 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5115
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5116 return (PyObject*) pad(self, 0, width - self->length, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5117 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5118
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5119 PyDoc_STRVAR(lower__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5120 "S.lower() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5121 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5122 Return a copy of the string S converted to lowercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5123
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5124 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5125 unicode_lower(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5126 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5127 return fixup(self, fixlower);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5128 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5129
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5130 #define LEFTSTRIP 0
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5131 #define RIGHTSTRIP 1
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5132 #define BOTHSTRIP 2
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5133
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5134 /* Arrays indexed by above */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5135 static const char *stripformat[] = {"|O:lstrip", "|O:rstrip", "|O:strip"};
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5136
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5137 #define STRIPNAME(i) (stripformat[i]+3)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5138
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5139 static const Py_UNICODE *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5140 unicode_memchr(const Py_UNICODE *s, Py_UNICODE c, size_t n)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5141 {
23228
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which
Tim Peters <tim.peters@gmail.com>
parents: 23225
diff changeset
5142 size_t i;
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which
Tim Peters <tim.peters@gmail.com>
parents: 23225
diff changeset
5143 for (i = 0; i < n; ++i)
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which
Tim Peters <tim.peters@gmail.com>
parents: 23225
diff changeset
5144 if (s[i] == c)
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5145 return s+i;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5146 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5147 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5148
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5149 /* externally visible for str.strip(unicode) */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5150 PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5151 _PyUnicode_XStrip(PyUnicodeObject *self, int striptype, PyObject *sepobj)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5152 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5153 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5154 int len = PyUnicode_GET_SIZE(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5155 Py_UNICODE *sep = PyUnicode_AS_UNICODE(sepobj);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5156 int seplen = PyUnicode_GET_SIZE(sepobj);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5157 int i, j;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5158
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5159 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5160 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5161 while (i < len && unicode_memchr(sep, s[i], seplen)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5162 i++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5163 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5164 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5165
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5166 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5167 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5168 do {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5169 j--;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5170 } while (j >= i && unicode_memchr(sep, s[j], seplen));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5171 j++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5172 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5173
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5174 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5175 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5176 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5177 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5178 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5179 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5180 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5181
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5182
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5183 static PyObject *
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5184 do_strip(PyUnicodeObject *self, int striptype)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5185 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5186 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5187 int len = PyUnicode_GET_SIZE(self), i, j;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5188
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5189 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5190 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5191 while (i < len && Py_UNICODE_ISSPACE(s[i])) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5192 i++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5193 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5194 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5195
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5196 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5197 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5198 do {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5199 j--;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5200 } while (j >= i && Py_UNICODE_ISSPACE(s[j]));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5201 j++;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5202 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5203
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5204 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5205 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5206 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5207 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5208 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5209 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5210 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5211
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5212
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5213 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5214 do_argstrip(PyUnicodeObject *self, int striptype, PyObject *args)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5215 {
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5216 PyObject *sep = NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5217
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5218 if (!PyArg_ParseTuple(args, (char *)stripformat[striptype], &sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5219 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5220
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5221 if (sep != NULL && sep != Py_None) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5222 if (PyUnicode_Check(sep))
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5223 return _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5224 else if (PyString_Check(sep)) {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5225 PyObject *res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5226 sep = PyUnicode_FromObject(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5227 if (sep==NULL)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5228 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5229 res = _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5230 Py_DECREF(sep);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5231 return res;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5232 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5233 else {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5234 PyErr_Format(PyExc_TypeError,
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5235 "%s arg must be None, unicode or str",
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5236 STRIPNAME(striptype));
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5237 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5238 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5239 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5240
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5241 return do_strip(self, striptype);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5242 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5243
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5244
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5245 PyDoc_STRVAR(strip__doc__,
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5246 "S.strip([sep]) -> unicode\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5247 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5248 Return a copy of the string S with leading and trailing\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5249 whitespace removed.\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5250 If sep is given and not None, remove characters in sep instead.\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5251 If sep is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5252
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5253 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5254 unicode_strip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5255 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5256 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5257 return do_strip(self, BOTHSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5258 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5259 return do_argstrip(self, BOTHSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5260 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5261
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5262
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5263 PyDoc_STRVAR(lstrip__doc__,
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5264 "S.lstrip([sep]) -> unicode\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5265 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5266 Return a copy of the string S with leading whitespace removed.\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5267 If sep is given and not None, remove characters in sep instead.\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5268 If sep is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5269
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5270 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5271 unicode_lstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5272 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5273 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5274 return do_strip(self, LEFTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5275 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5276 return do_argstrip(self, LEFTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5277 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5278
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5279
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5280 PyDoc_STRVAR(rstrip__doc__,
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5281 "S.rstrip([sep]) -> unicode\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5282 \n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5283 Return a copy of the string S with trailing whitespace removed.\n\
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5284 If sep is given and not None, remove characters in sep instead.\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5285 If sep is a str, it will be converted to unicode before stripping");
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5286
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5287 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5288 unicode_rstrip(PyUnicodeObject *self, PyObject *args)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5289 {
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5290 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5291 return do_strip(self, RIGHTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5292 else
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5293 return do_argstrip(self, RIGHTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5294 }
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5295
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5296
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5297 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5298 unicode_repeat(PyUnicodeObject *str, int len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5299 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5300 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5301 Py_UNICODE *p;
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5302 int nchars;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5303 size_t nbytes;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5304
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5305 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5306 len = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5307
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5308 if (len == 1 && PyUnicode_CheckExact(str)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5309 /* no repeat, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5310 Py_INCREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5311 return (PyObject*) str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5312 }
14492
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5313
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5314 /* ensure # of chars needed doesn't overflow int and # of bytes
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5315 * needed doesn't overflow size_t
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5316 */
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5317 nchars = len * str->length;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5318 if (len && nchars / len != str->length) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5319 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5320 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5321 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5322 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5323 nbytes = (nchars + 1) * sizeof(Py_UNICODE);
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5324 if (nbytes / sizeof(Py_UNICODE) != (size_t)(nchars + 1)) {
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5325 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5326 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5327 return NULL;
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5328 }
7728b0ea9344 Fix for bug 113934. string*n and unicode*n did no overflow checking at
Tim Peters <tim.peters@gmail.com>
parents: 14388
diff changeset
5329 u = _PyUnicode_New(nchars);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5330 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5331 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5332
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5333 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5334
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5335 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5336 Py_UNICODE_COPY(p, str->str, str->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5337 p += str->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5338 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5340 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5341 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5342
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5343 PyObject *PyUnicode_Replace(PyObject *obj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5344 PyObject *subobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5345 PyObject *replobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5346 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5347 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5348 PyObject *self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5349 PyObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5350 PyObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5351 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5352
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5353 self = PyUnicode_FromObject(obj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5354 if (self == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5355 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5356 str1 = PyUnicode_FromObject(subobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5357 if (str1 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5358 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5359 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5360 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5361 str2 = PyUnicode_FromObject(replobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5362 if (str2 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5363 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5364 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5365 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5366 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5367 result = replace((PyUnicodeObject *)self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5368 (PyUnicodeObject *)str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5369 (PyUnicodeObject *)str2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5370 maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5371 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5372 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5373 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5374 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5375 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5376
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5377 PyDoc_STRVAR(replace__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5378 "S.replace (old, new[, maxsplit]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5379 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5380 Return a copy of S with all occurrences of substring\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5381 old replaced by new. If the optional argument maxsplit is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5382 given, only the first maxsplit occurrences are replaced.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5383
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5384 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5385 unicode_replace(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5386 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5387 PyUnicodeObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5388 PyUnicodeObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5389 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5390 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5391
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5392 if (!PyArg_ParseTuple(args, "OO|i:replace", &str1, &str2, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5393 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5394 str1 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5395 if (str1 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5396 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5397 str2 = (PyUnicodeObject *)PyUnicode_FromObject((PyObject *)str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5398 if (str2 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5399 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5400
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5401 result = replace(self, str1, str2, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5402
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5403 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5404 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5405 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5406 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5407
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5408 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5409 PyObject *unicode_repr(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5410 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5411 return unicodeescape_string(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5412 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5413 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5414 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5415
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5416 PyDoc_STRVAR(rfind__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5417 "S.rfind(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5418 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5419 Return the highest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5420 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5421 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5422 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5423 Return -1 on failure.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5424
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5425 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5426 unicode_rfind(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5427 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5428 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5429 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5430 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5431 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5432
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5433 if (!PyArg_ParseTuple(args, "O|O&O&:rfind", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5434 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5435 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5436 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5437 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5438 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5439 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5440
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5441 result = PyInt_FromLong(findstring(self, substring, start, end, -1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5442
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5443 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5444 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5445 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5446
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5447 PyDoc_STRVAR(rindex__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5448 "S.rindex(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5449 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5450 Like S.rfind() but raise ValueError when the substring is not found.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5451
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5452 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5453 unicode_rindex(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5454 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5455 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5456 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5457 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5458 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5459
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5460 if (!PyArg_ParseTuple(args, "O|O&O&:rindex", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5461 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5462 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5463 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5464 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5465 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5466 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5467
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5468 result = findstring(self, substring, start, end, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5469
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5470 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5471 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5472 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5473 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5474 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5475 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5476 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5477
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5478 PyDoc_STRVAR(rjust__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5479 "S.rjust(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5480 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5481 Return S right justified in a Unicode string of length width. Padding is\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5482 done using spaces.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5483
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5484 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5485 unicode_rjust(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5486 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5487 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5488 if (!PyArg_ParseTuple(args, "i:rjust", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5489 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5490
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5491 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5492 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5493 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5494 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5495
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5496 return (PyObject*) pad(self, width - self->length, 0, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5497 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5498
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5499 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5500 unicode_slice(PyUnicodeObject *self, int start, int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5501 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5502 /* standard clamping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5503 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5504 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5505 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5506 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5507 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5508 end = self->length;
19885
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses.
Tim Peters <tim.peters@gmail.com>
parents: 19849
diff changeset
5509 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5510 /* full slice, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5511 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5512 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5513 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5514 if (start > end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5515 start = end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5516 /* copy slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5517 return (PyObject*) PyUnicode_FromUnicode(self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5518 end - start);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5519 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5520
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5521 PyObject *PyUnicode_Split(PyObject *s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5522 PyObject *sep,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5523 int maxsplit)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5524 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5525 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5526
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5527 s = PyUnicode_FromObject(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5528 if (s == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5529 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5530 if (sep != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5531 sep = PyUnicode_FromObject(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5532 if (sep == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5533 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5534 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5535 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5536 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5537
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5538 result = split((PyUnicodeObject *)s, (PyUnicodeObject *)sep, maxsplit);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5539
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5540 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5541 Py_XDECREF(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5542 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5543 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5544
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5545 PyDoc_STRVAR(split__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5546 "S.split([sep [,maxsplit]]) -> list of strings\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5547 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5548 Return a list of the words in S, using sep as the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5549 delimiter string. If maxsplit is given, at most maxsplit\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5550 splits are done. If sep is not specified, any whitespace string\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5551 is a separator.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5552
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5553 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5554 unicode_split(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5555 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5556 PyObject *substring = Py_None;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5557 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5558
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5559 if (!PyArg_ParseTuple(args, "|Oi:split", &substring, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5560 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5561
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5562 if (substring == Py_None)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5563 return split(self, NULL, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5564 else if (PyUnicode_Check(substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5565 return split(self, (PyUnicodeObject *)substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5566 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5567 return PyUnicode_Split((PyObject *)self, substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5568 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5569
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5570 PyDoc_STRVAR(splitlines__doc__,
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5571 "S.splitlines([keepends]]) -> list of strings\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5572 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5573 Return a list of the lines in S, breaking at line boundaries.\n\
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5574 Line breaks are not included in the resulting list unless keepends\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5575 is given and true.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5576
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5577 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5578 unicode_splitlines(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5579 {
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5580 int keepends = 0;
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5581
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5582 if (!PyArg_ParseTuple(args, "|i:splitlines", &keepends))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5583 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5584
12369
68e48bb94071 Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12329
diff changeset
5585 return PyUnicode_Splitlines((PyObject *)self, keepends);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5586 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5587
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5588 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5589 PyObject *unicode_str(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5590 {
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
5591 return PyUnicode_AsEncodedString((PyObject *)self, NULL, NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5592 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5593
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5594 PyDoc_STRVAR(swapcase__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5595 "S.swapcase() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5596 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5597 Return a copy of S with uppercase characters converted to lowercase\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5598 and vice versa.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5599
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5600 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5601 unicode_swapcase(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5602 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5603 return fixup(self, fixswapcase);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5604 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5605
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5606 PyDoc_STRVAR(translate__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5607 "S.translate(table) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5608 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5609 Return a copy of the string S, where all characters have been mapped\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5610 through the given translation table, which must be a mapping of\n\
25120
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
5611 Unicode ordinals to Unicode ordinals, Unicode strings or None.\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
5612 Unmapped characters are left untouched. Characters mapped to None\n\
953ebe0c0c25 Change the unicode.translate docstring to document that
Walter Dörwald <walter@livinglogic.de>
parents: 25091
diff changeset
5613 are deleted.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5614
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5615 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5616 unicode_translate(PyUnicodeObject *self, PyObject *table)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5617 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5618 return PyUnicode_TranslateCharmap(self->str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5619 self->length,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5620 table,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5621 "ignore");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5622 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5623
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5624 PyDoc_STRVAR(upper__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5625 "S.upper() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5626 \n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5627 Return a copy of S converted to uppercase.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5628
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5629 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5630 unicode_upper(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5631 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5632 return fixup(self, fixupper);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5633 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5634
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5635 PyDoc_STRVAR(zfill__doc__,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5636 "S.zfill(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5637 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5638 Pad a numeric string x with zeros on the left, to fill a field\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5639 of the specified width. The string x is never truncated.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5640
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5641 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5642 unicode_zfill(PyUnicodeObject *self, PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5643 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5644 int fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5645 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5646
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5647 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5648 if (!PyArg_ParseTuple(args, "i:zfill", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5649 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5650
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5651 if (self->length >= width) {
23051
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5652 if (PyUnicode_CheckExact(self)) {
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5653 Py_INCREF(self);
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5654 return (PyObject*) self;
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5655 }
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5656 else
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5657 return PyUnicode_FromUnicode(
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5658 PyUnicode_AS_UNICODE(self),
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5659 PyUnicode_GET_SIZE(self)
19ea00feda33 Return the orginal string only if it's a real str or unicode
Walter Dörwald <walter@livinglogic.de>
parents: 23045
diff changeset
5660 );
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5661 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5662
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5663 fill = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5664
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5665 u = pad(self, fill, 0, '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5666
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
5667 if (u == NULL)
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
5668 return NULL;
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
5669
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5670 if (u->str[fill] == '+' || u->str[fill] == '-') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5671 /* move sign to beginning of string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5672 u->str[0] = u->str[fill];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5673 u->str[fill] = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5674 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5675
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5676 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5677 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5678
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5679 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5680 static PyObject*
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5681 unicode_freelistsize(PyUnicodeObject *self)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5682 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5683 return PyInt_FromLong(unicode_freelist_size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5684 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5685 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5686
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5687 PyDoc_STRVAR(startswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5688 "S.startswith(prefix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5689 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5690 Return True if S starts with the specified prefix, False otherwise. With\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5691 optional start, test S beginning at that position. With optional end, stop\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5692 comparing S at that position.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5693
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5694 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5695 unicode_startswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5696 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5697 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5698 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5699 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5700 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5701 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5702
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5703 if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5704 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5705 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5706 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5707 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5708 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5709 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5710
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5711 result = PyBool_FromLong(tailmatch(self, substring, start, end, -1));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5712
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5713 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5714 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5715 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5716
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5717
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5718 PyDoc_STRVAR(endswith__doc__,
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5719 "S.endswith(suffix[, start[, end]]) -> bool\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5720 \n\
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5721 Return True if S ends with the specified suffix, False otherwise. With\n\
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5722 optional start, test S beginning at that position. With optional end, stop\n\
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
5723 comparing S at that position.");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5724
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5725 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5726 unicode_endswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5727 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5728 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5729 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5730 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5731 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5732 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5733
12629
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5734 if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &substring,
3efeaa7521e4 Trent Mick:
Guido van Rossum <guido@python.org>
parents: 12582
diff changeset
5735 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5736 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5737 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5738 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5739 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5740 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5741
22785
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in
Guido van Rossum <guido@python.org>
parents: 22521
diff changeset
5742 result = PyBool_FromLong(tailmatch(self, substring, start, end, +1));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5743
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5744 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5745 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5746 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5747
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5748
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5749 static PyMethodDef unicode_methods[] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5750
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5751 /* Order is according to common usage: often used methods should
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5752 appear first, since lookup is done sequentially. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5753
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5754 {"encode", (PyCFunction) unicode_encode, METH_VARARGS, encode__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5755 {"replace", (PyCFunction) unicode_replace, METH_VARARGS, replace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5756 {"split", (PyCFunction) unicode_split, METH_VARARGS, split__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5757 {"join", (PyCFunction) unicode_join, METH_O, join__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5758 {"capitalize", (PyCFunction) unicode_capitalize, METH_NOARGS, capitalize__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5759 {"title", (PyCFunction) unicode_title, METH_NOARGS, title__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5760 {"center", (PyCFunction) unicode_center, METH_VARARGS, center__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5761 {"count", (PyCFunction) unicode_count, METH_VARARGS, count__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5762 {"expandtabs", (PyCFunction) unicode_expandtabs, METH_VARARGS, expandtabs__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5763 {"find", (PyCFunction) unicode_find, METH_VARARGS, find__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5764 {"index", (PyCFunction) unicode_index, METH_VARARGS, index__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5765 {"ljust", (PyCFunction) unicode_ljust, METH_VARARGS, ljust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5766 {"lower", (PyCFunction) unicode_lower, METH_NOARGS, lower__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5767 {"lstrip", (PyCFunction) unicode_lstrip, METH_VARARGS, lstrip__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5768 /* {"maketrans", (PyCFunction) unicode_maketrans, METH_VARARGS, maketrans__doc__}, */
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5769 {"rfind", (PyCFunction) unicode_rfind, METH_VARARGS, rfind__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5770 {"rindex", (PyCFunction) unicode_rindex, METH_VARARGS, rindex__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5771 {"rjust", (PyCFunction) unicode_rjust, METH_VARARGS, rjust__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5772 {"rstrip", (PyCFunction) unicode_rstrip, METH_VARARGS, rstrip__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5773 {"splitlines", (PyCFunction) unicode_splitlines, METH_VARARGS, splitlines__doc__},
23225
e639e34ef901 Apply patch diff.txt from SF feature request
Walter Dörwald <walter@livinglogic.de>
parents: 23205
diff changeset
5774 {"strip", (PyCFunction) unicode_strip, METH_VARARGS, strip__doc__},
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5775 {"swapcase", (PyCFunction) unicode_swapcase, METH_NOARGS, swapcase__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5776 {"translate", (PyCFunction) unicode_translate, METH_O, translate__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5777 {"upper", (PyCFunction) unicode_upper, METH_NOARGS, upper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5778 {"startswith", (PyCFunction) unicode_startswith, METH_VARARGS, startswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5779 {"endswith", (PyCFunction) unicode_endswith, METH_VARARGS, endswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5780 {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5781 {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5782 {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5783 {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5784 {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5785 {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5786 {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5787 {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5788 {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
23045
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241
Walter Dörwald <walter@livinglogic.de>
parents: 22981
diff changeset
5789 {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5790 #if 0
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5791 {"capwords", (PyCFunction) unicode_capwords, METH_NOARGS, capwords__doc__},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5792 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5793
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5794 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5795 /* This one is just used for debugging the implementation. */
19301
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O.
Martin v. Löwis <martin@v.loewis.de>
parents: 19121
diff changeset
5796 {"freelistsize", (PyCFunction) unicode_freelistsize, METH_NOARGS},
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5797 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5798
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5799 {NULL, NULL}
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5800 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5801
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5802 static PyObject *
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5803 unicode_mod(PyObject *v, PyObject *w)
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5804 {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5805 if (!PyUnicode_Check(v)) {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5806 Py_INCREF(Py_NotImplemented);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5807 return Py_NotImplemented;
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5808 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5809 return PyUnicode_Format(v, w);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5810 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5811
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5812 static PyNumberMethods unicode_as_number = {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5813 0, /*nb_add*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5814 0, /*nb_subtract*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5815 0, /*nb_multiply*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5816 0, /*nb_divide*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5817 unicode_mod, /*nb_remainder*/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5818 };
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
5819
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5820 static PySequenceMethods unicode_as_sequence = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5821 (inquiry) unicode_length, /* sq_length */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5822 (binaryfunc) PyUnicode_Concat, /* sq_concat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5823 (intargfunc) unicode_repeat, /* sq_repeat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5824 (intargfunc) unicode_getitem, /* sq_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5825 (intintargfunc) unicode_slice, /* sq_slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5826 0, /* sq_ass_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5827 0, /* sq_ass_slice */
11997
38310eb0659e Add sq_contains implementation.
Guido van Rossum <guido@python.org>
parents: 11968
diff changeset
5828 (objobjproc)PyUnicode_Contains, /*sq_contains*/
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5829 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5830
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5831 static PyObject*
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5832 unicode_subscript(PyUnicodeObject* self, PyObject* item)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5833 {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5834 if (PyInt_Check(item)) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5835 long i = PyInt_AS_LONG(item);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5836 if (i < 0)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5837 i += PyString_GET_SIZE(self);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5838 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5839 } else if (PyLong_Check(item)) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5840 long i = PyLong_AsLong(item);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5841 if (i == -1 && PyErr_Occurred())
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5842 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5843 if (i < 0)
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5844 i += PyString_GET_SIZE(self);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5845 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5846 } else if (PySlice_Check(item)) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5847 int start, stop, step, slicelength, cur, i;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5848 Py_UNICODE* source_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5849 Py_UNICODE* result_buf;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5850 PyObject* result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5851
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5852 if (PySlice_GetIndicesEx((PySliceObject*)item, PyString_GET_SIZE(self),
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5853 &start, &stop, &step, &slicelength) < 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5854 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5855 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5856
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5857 if (slicelength <= 0) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5858 return PyUnicode_FromUnicode(NULL, 0);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5859 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5860 source_buf = PyUnicode_AS_UNICODE((PyObject*)self);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5861 result_buf = PyMem_MALLOC(slicelength*sizeof(Py_UNICODE));
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5862
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5863 for (cur = start, i = 0; i < slicelength; cur += step, i++) {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5864 result_buf[i] = source_buf[cur];
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5865 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5866
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5867 result = PyUnicode_FromUnicode(result_buf, slicelength);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5868 PyMem_FREE(result_buf);
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5869 return result;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5870 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5871 } else {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5872 PyErr_SetString(PyExc_TypeError, "string indices must be integers");
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5873 return NULL;
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5874 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5875 }
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5876
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5877 static PyMappingMethods unicode_as_mapping = {
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5878 (inquiry)unicode_length, /* mp_length */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5879 (binaryfunc)unicode_subscript, /* mp_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5880 (objobjargproc)0, /* mp_ass_subscript */
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5881 };
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
5882
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5883 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5884 unicode_buffer_getreadbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5885 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5886 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5887 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5888 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5889 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5890 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5891 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5892 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5893 *ptr = (void *) self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5894 return PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5895 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5896
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5897 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5898 unicode_buffer_getwritebuf(PyUnicodeObject *self, int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5899 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5900 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5901 PyErr_SetString(PyExc_TypeError,
23837
359a57e96d5e Fix typo in exception message
Neal Norwitz <nnorwitz@gmail.com>
parents: 23832
diff changeset
5902 "cannot use unicode as modifiable buffer");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5903 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5904 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5905
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5906 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5907 unicode_buffer_getsegcount(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5908 int *lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5909 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5910 if (lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5911 *lenp = PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5912 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5913 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5914
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5915 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5916 unicode_buffer_getcharbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5917 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5918 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5919 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5920 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5921
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5922 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5923 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5924 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5925 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5926 }
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
5927 str = _PyUnicode_AsDefaultEncodedString((PyObject *)self, NULL);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5928 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5929 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5930 *ptr = (void *) PyString_AS_STRING(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5931 return PyString_GET_SIZE(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5932 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5933
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5934 /* Helpers for PyUnicode_Format() */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5935
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5936 static PyObject *
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
5937 getnextarg(PyObject *args, int arglen, int *p_argidx)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5938 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5939 int argidx = *p_argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5940 if (argidx < arglen) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5941 (*p_argidx)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5942 if (arglen < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5943 return args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5944 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5945 return PyTuple_GetItem(args, argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5946 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5947 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5948 "not enough arguments for format string");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5949 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5950 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5951
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5952 #define F_LJUST (1<<0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5953 #define F_SIGN (1<<1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5954 #define F_BLANK (1<<2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5955 #define F_ALT (1<<3)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5956 #define F_ZERO (1<<4)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5957
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5958 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5959 int usprintf(register Py_UNICODE *buffer, char *format, ...)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5960 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5961 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5962 int len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5963 va_list va;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5964 char *charbuffer;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5965 va_start(va, format);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5966
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5967 /* First, format the string as char array, then expand to Py_UNICODE
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5968 array. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5969 charbuffer = (char *)buffer;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5970 len = vsprintf(charbuffer, format, va);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5971 for (i = len - 1; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5972 buffer[i] = (Py_UNICODE) charbuffer[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5973
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5974 va_end(va);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5975 return len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5976 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5977
24736
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
5978 /* XXX To save some code duplication, formatfloat/long/int could have been
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
5979 shared with stringobject.c, converting from 8-bit to Unicode after the
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
5980 formatting is done. */
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
5981
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5982 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5983 formatfloat(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
5984 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5985 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5986 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5987 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5988 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5989 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
5990 /* fmt = '%#.' + `prec` + `type`
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
5991 worst case length = 3 + 10 (len of INT_MAX) + 1 = 14 (use 20)*/
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5992 char fmt[20];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5993 double x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5994
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5995 x = PyFloat_AsDouble(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5996 if (x == -1.0 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5997 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5998 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
5999 prec = 6;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6000 if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6001 type = 'g';
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6002 /* Worst case length calc to ensure no buffer overrun:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6003
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6004 'g' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6005 fmt = %#.<prec>g
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6006 buf = '-' + [0-9]*prec + '.' + 'e+' + (longest exp
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6007 for any double rep.)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6008 len = 1 + prec + 1 + 2 + 5 = 9 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6009
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6010 'f' formats:
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6011 buf = '-' + [0-9]*x + '.' + [0-9]*prec (with x < 50)
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6012 len = 1 + 50 + 1 + prec = 52 + prec
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6013
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6014 If prec=0 the effective precision is 1 (the leading digit is
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6015 always given), therefore increase the length by one.
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6016
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6017 */
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6018 if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) ||
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6019 (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) {
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6020 PyErr_SetString(PyExc_OverflowError,
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6021 "formatted float is too long (precision too large?)");
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6022 return -1;
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6023 }
21082
c8d775b9ff92 formatfloat(), formatint(): Conversion of sprintf() to PyOS_snprintf()
Barry Warsaw <barry@python.org>
parents: 21067
diff changeset
6024 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%d%c",
26825
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6025 (flags&F_ALT) ? "#" : "",
e153961f961f Patch for bug #659709: bogus computation of float length
Marc-André Lemburg <mal@egenix.com>
parents: 26274
diff changeset
6026 prec, type);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6027 return usprintf(buf, fmt, x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6028 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6029
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6030 static PyObject*
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6031 formatlong(PyObject *val, int flags, int prec, int type)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6032 {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6033 char *buf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6034 int i, len;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6035 PyObject *str; /* temporary string object. */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6036 PyUnicodeObject *result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6037
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6038 str = _PyString_FormatLong(val, flags, prec, type, &buf, &len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6039 if (!str)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6040 return NULL;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6041 result = _PyUnicode_New(len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6042 for (i = 0; i < len; i++)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6043 result->str[i] = buf[i];
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6044 result->str[len] = 0;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6045 Py_DECREF(str);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6046 return (PyObject*)result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6047 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6048
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6049 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6050 formatint(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6051 size_t buflen,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6052 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6053 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6054 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6055 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6056 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6057 /* fmt = '%#.' + `prec` + 'l' + `type`
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6058 * worst case length = 3 + 19 (worst len of INT_MAX on 64-bit machine)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6059 * + 1 + 1
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6060 * = 24
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6061 */
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6062 char fmt[64]; /* plenty big enough! */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6063 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6064
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6065 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6066 if (x == -1 && PyErr_Occurred())
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6067 return -1;
24736
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6068 if (x < 0 && type != 'd' && type != 'i') {
24831
a602e4878b4e More changes of DeprecationWarning to FutureWarning.
Guido van Rossum <guido@python.org>
parents: 24737
diff changeset
6069 if (PyErr_Warn(PyExc_FutureWarning,
24736
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6070 "%u/%o/%x/%X of negative int will return "
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6071 "a signed string in Python 2.4 and up") < 0)
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6072 return -1;
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that
Guido van Rossum <guido@python.org>
parents: 24703
diff changeset
6073 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6074 if (prec < 0)
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6075 prec = 1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6076
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6077 /* buf = '+'/'-'/'0'/'0x' + '[0-9]'*max(prec,len(x in octal))
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6078 * worst case buf = '0x' + [0-9]*prec, where prec >= 11
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6079 */
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6080 if (buflen <= 13 || buflen <= (size_t)2 + (size_t)prec) {
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6081 PyErr_SetString(PyExc_OverflowError,
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6082 "formatted integer is too long (precision too large?)");
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6083 return -1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6084 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6085
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6086 if ((flags & F_ALT) &&
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6087 (type == 'x' || type == 'X')) {
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6088 /* When converting under %#x or %#X, there are a number
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6089 * of issues that cause pain:
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6090 * - when 0 is being converted, the C standard leaves off
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6091 * the '0x' or '0X', which is inconsistent with other
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6092 * %#x/%#X conversions and inconsistent with Python's
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6093 * hex() function
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6094 * - there are platforms that violate the standard and
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6095 * convert 0 with the '0x' or '0X'
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6096 * (Metrowerks, Compaq Tru64)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6097 * - there are platforms that give '0x' when converting
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6098 * under %#X, but convert 0 in accordance with the
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6099 * standard (OS/2 EMX)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6100 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6101 * We can achieve the desired consistency by inserting our
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6102 * own '0x' or '0X' prefix, and substituting %x/%X in place
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6103 * of %#x/%#X.
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6104 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6105 * Note that this is the same approach as used in
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6106 * formatint() in stringobject.c
22053
49197fd76362 OS/2 EMX port changes (Objects part of patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 21886
diff changeset
6107 */
22087
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6108 PyOS_snprintf(fmt, sizeof(fmt), "0%c%%.%dl%c",
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6109 type, prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6110 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6111 else {
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6112 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%dl%c",
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6113 (flags&F_ALT) ? "#" : "",
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6114 prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267):
Andrew MacIntyre <andymac@bullseye.apana.org.au>
parents: 22053
diff changeset
6115 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6116 return usprintf(buf, fmt, x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6117 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6118
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6119 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6120 formatchar(Py_UNICODE *buf,
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6121 size_t buflen,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6122 PyObject *v)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6123 {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6124 /* presume that the buffer is at least 2 characters long */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6125 if (PyUnicode_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6126 if (PyUnicode_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6127 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6128 buf[0] = PyUnicode_AS_UNICODE(v)[0];
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6129 }
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6130
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6131 else if (PyString_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6132 if (PyString_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6133 goto onError;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6134 buf[0] = (Py_UNICODE)PyString_AS_STRING(v)[0];
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6135 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6136
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6137 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6138 /* Integer input truncated to a character */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6139 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6140 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6141 if (x == -1 && PyErr_Occurred())
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6142 goto onError;
24737
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6143 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6144 if (x < 0 || x > 0x10ffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6145 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6146 "%c arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6147 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6148 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6149 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6150 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6151 if (x < 0 || x > 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6152 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6153 "%c arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6154 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6155 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6156 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6157 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level.
Marc-André Lemburg <mal@egenix.com>
parents: 24736
diff changeset
6158 buf[0] = (Py_UNICODE) x;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6159 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6160 buf[1] = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6161 return 1;
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6162
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6163 onError:
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6164 PyErr_SetString(PyExc_TypeError,
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6165 "%c requires int or char");
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6166 return -1;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6167 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6168
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6169 /* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6170
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6171 FORMATBUFLEN is the length of the buffer in which the floats, ints, &
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6172 chars are formatted. XXX This is a magic number. Each formatting
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6173 routine does bounds checking to ensure no overflow, but a better
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6174 solution may be to malloc a buffer of appropriate size for each
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6175 format. For now, the current solution is sufficient.
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6176 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6177 #define FORMATBUFLEN (size_t)120
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6178
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6179 PyObject *PyUnicode_Format(PyObject *format,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6180 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6181 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6182 Py_UNICODE *fmt, *res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6183 int fmtcnt, rescnt, reslen, arglen, argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6184 int args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6185 PyUnicodeObject *result = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6186 PyObject *dict = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6187 PyObject *uformat;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6188
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6189 if (format == NULL || args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6190 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6191 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6192 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6193 uformat = PyUnicode_FromObject(format);
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6194 if (uformat == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6195 return NULL;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6196 fmt = PyUnicode_AS_UNICODE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6197 fmtcnt = PyUnicode_GET_SIZE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6198
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6199 reslen = rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6200 result = _PyUnicode_New(reslen);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6201 if (result == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6202 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6203 res = PyUnicode_AS_UNICODE(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6204
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6205 if (PyTuple_Check(args)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6206 arglen = PyTuple_Size(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6207 argidx = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6208 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6209 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6210 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6211 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6212 }
26164
63a08639dae2 Fix SF # 635969, No error "not all arguments converted"
Neal Norwitz <nnorwitz@gmail.com>
parents: 25895
diff changeset
6213 if (args->ob_type->tp_as_mapping && !PyTuple_Check(args) &&
63a08639dae2 Fix SF # 635969, No error "not all arguments converted"
Neal Norwitz <nnorwitz@gmail.com>
parents: 25895
diff changeset
6214 !PyObject_TypeCheck(args, &PyBaseString_Type))
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6215 dict = args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6216
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6217 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6218 if (*fmt != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6219 if (--rescnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6220 rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6221 reslen += rescnt;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6222 if (_PyUnicode_Resize(&result, reslen) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6223 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6224 res = PyUnicode_AS_UNICODE(result) + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6225 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6226 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6227 *res++ = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6228 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6229 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6230 /* Got a format specifier */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6231 int flags = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6232 int width = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6233 int prec = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6234 Py_UNICODE c = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6235 Py_UNICODE fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6236 PyObject *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6237 PyObject *temp = NULL;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6238 Py_UNICODE *pbuf;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6239 Py_UNICODE sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6240 int len;
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6241 Py_UNICODE formatbuf[FORMATBUFLEN]; /* For format{float,int,char}() */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6242
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6243 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6244 if (*fmt == '(') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6245 Py_UNICODE *keystart;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6246 int keylen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6247 PyObject *key;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6248 int pcount = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6249
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6250 if (dict == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6251 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6252 "format requires a mapping");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6253 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6254 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6255 ++fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6256 --fmtcnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6257 keystart = fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6258 /* Skip over balanced parentheses */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6259 while (pcount > 0 && --fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6260 if (*fmt == ')')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6261 --pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6262 else if (*fmt == '(')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6263 ++pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6264 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6265 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6266 keylen = fmt - keystart - 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6267 if (fmtcnt < 0 || pcount > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6268 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6269 "incomplete format key");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6270 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6271 }
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6272 #if 0
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6273 /* keys are converted to strings using UTF-8 and
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6274 then looked up since Python uses strings to hold
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6275 variables names etc. in its namespaces and we
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6276 wouldn't want to break common idioms. */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6277 key = PyUnicode_EncodeUTF8(keystart,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6278 keylen,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6279 NULL);
21028
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6280 #else
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6281 key = PyUnicode_FromUnicode(keystart, keylen);
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects.
Marc-André Lemburg <mal@egenix.com>
parents: 20563
diff changeset
6282 #endif
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6283 if (key == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6284 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6285 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6286 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6287 args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6288 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6289 args = PyObject_GetItem(dict, key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6290 Py_DECREF(key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6291 if (args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6292 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6293 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6294 args_owned = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6295 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6296 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6297 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6298 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6299 switch (c = *fmt++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6300 case '-': flags |= F_LJUST; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6301 case '+': flags |= F_SIGN; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6302 case ' ': flags |= F_BLANK; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6303 case '#': flags |= F_ALT; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6304 case '0': flags |= F_ZERO; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6305 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6306 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6307 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6308 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6309 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6310 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6311 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6312 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6313 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6314 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6315 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6316 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6317 width = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6318 if (width < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6319 flags |= F_LJUST;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6320 width = -width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6321 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6322 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6323 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6324 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6325 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6326 width = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6327 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6328 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6329 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6330 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6331 if ((width*10) / 10 != width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6332 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6333 "width too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6334 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6335 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6336 width = width*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6337 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6338 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6339 if (c == '.') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6340 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6341 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6342 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6343 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6344 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6345 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6346 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6347 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6348 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6349 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6350 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6351 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6352 prec = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6353 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6354 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6355 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6356 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6357 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6358 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6359 prec = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6360 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6361 c = Py_CHARMASK(*fmt++);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6362 if (c < '0' || c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6363 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6364 if ((prec*10) / 10 != prec) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6365 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6366 "prec too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6367 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6368 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6369 prec = prec*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6370 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6371 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6372 } /* prec */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6373 if (fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6374 if (c == 'h' || c == 'l' || c == 'L') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6375 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6376 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6377 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6378 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6379 if (fmtcnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6380 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6381 "incomplete format");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6382 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6383 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6384 if (c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6385 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6386 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6387 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6388 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6389 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6390 fill = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6391 switch (c) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6392
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6393 case '%':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6394 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6395 /* presume that buffer length is at least 1 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6396 pbuf[0] = '%';
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6397 len = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6398 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6399
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6400 case 's':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6401 case 'r':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6402 if (PyUnicode_Check(v) && c == 's') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6403 temp = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6404 Py_INCREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6405 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6406 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6407 PyObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6408 if (c == 's')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6409 temp = PyObject_Str(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6410 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6411 temp = PyObject_Repr(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6412 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6413 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6414 if (!PyString_Check(temp)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6415 /* XXX Note: this should never happen, since
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6416 PyObject_Repr() and PyObject_Str() assure
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6417 this */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6418 Py_DECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6419 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6420 "%s argument has non-string str()");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6421 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6422 }
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6423 unicode = PyUnicode_Decode(PyString_AS_STRING(temp),
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6424 PyString_GET_SIZE(temp),
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6425 NULL,
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6426 "strict");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6427 Py_DECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6428 temp = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6429 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6430 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6431 }
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6432 pbuf = PyUnicode_AS_UNICODE(temp);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6433 len = PyUnicode_GET_SIZE(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6434 if (prec >= 0 && len > prec)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6435 len = prec;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6436 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6437
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6438 case 'i':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6439 case 'd':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6440 case 'u':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6441 case 'o':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6442 case 'x':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6443 case 'X':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6444 if (c == 'i')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6445 c = 'd';
15455
d0146d128897 Fox for SF bug #123859: %[duxXo] long formats inconsistent.
Tim Peters <tim.peters@gmail.com>
parents: 15001
diff changeset
6446 if (PyLong_Check(v)) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6447 temp = formatlong(v, flags, prec, c);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6448 if (!temp)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6449 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6450 pbuf = PyUnicode_AS_UNICODE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6451 len = PyUnicode_GET_SIZE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6452 /* unbounded ints can always produce
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6453 a sign character! */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6454 sign = 1;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6455 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6456 else {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6457 pbuf = formatbuf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6458 len = formatint(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6459 flags, prec, c, v);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6460 if (len < 0)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6461 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6462 /* only d conversion is signed */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6463 sign = c == 'd';
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6464 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6465 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6466 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6467 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6468
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6469 case 'e':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6470 case 'E':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6471 case 'f':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6472 case 'g':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6473 case 'G':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6474 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6475 len = formatfloat(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6476 flags, prec, c, v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6477 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6478 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6479 sign = 1;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6480 if (flags & F_ZERO)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6481 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6482 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6483
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6484 case 'c':
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6485 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6486 len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6487 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6488 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6489 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6490
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6491 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6492 PyErr_Format(PyExc_ValueError,
15579
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the
Andrew M. Kuchling <amk@amk.ca>
parents: 15455
diff changeset
6493 "unsupported format character '%c' (0x%x) "
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the
Andrew M. Kuchling <amk@amk.ca>
parents: 15455
diff changeset
6494 "at index %i",
25248
fc2b73802f7b Fix part of SF bug # 544248 gcc warning in unicodeobject.c
Neal Norwitz <nnorwitz@gmail.com>
parents: 25233
diff changeset
6495 (31<=c && c<=126) ? (char)c : '?',
25375
4df0e691f4da Add cast to avoid compiler warning.
Marc-André Lemburg <mal@egenix.com>
parents: 25248
diff changeset
6496 (int)c,
25233
8d751b62ecc4 Fix warnings on 64-bit platforms about casts from pointers to ints.
Guido van Rossum <guido@python.org>
parents: 25120
diff changeset
6497 (int)(fmt -1 - PyUnicode_AS_UNICODE(uformat)));
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6498 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6499 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6500 if (sign) {
13196
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6501 if (*pbuf == '-' || *pbuf == '+') {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 13110
diff changeset
6502 sign = *pbuf++;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6503 len--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6504 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6505 else if (flags & F_SIGN)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6506 sign = '+';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6507 else if (flags & F_BLANK)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6508 sign = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6509 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6510 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6511 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6512 if (width < len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6513 width = len;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
6514 if (rescnt - (sign != 0) < width) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6515 reslen -= rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6516 rescnt = width + fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6517 reslen += rescnt;
25719
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
6518 if (reslen < 0) {
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
6519 Py_DECREF(result);
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
6520 return PyErr_NoMemory();
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623:
Guido van Rossum <guido@python.org>
parents: 25375
diff changeset
6521 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6522 if (_PyUnicode_Resize(&result, reslen) < 0)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6523 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6524 res = PyUnicode_AS_UNICODE(result)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6525 + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6526 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6527 if (sign) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6528 if (fill != ' ')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6529 *res++ = sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6530 rescnt--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6531 if (width > len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6532 width--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6533 }
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6534 if ((flags & F_ALT) && (c == 'x' || c == 'X')) {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6535 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6536 assert(pbuf[1] == c);
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6537 if (fill != ' ') {
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6538 *res++ = *pbuf++;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6539 *res++ = *pbuf++;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6540 }
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6541 rescnt -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6542 width -= 2;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6543 if (width < 0)
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6544 width = 0;
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6545 len -= 2;
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6546 }
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6547 if (width > len && !(flags & F_LJUST)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6548 do {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6549 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6550 *res++ = fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6551 } while (--width > len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6552 }
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6553 if (fill == ' ') {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6554 if (sign)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6555 *res++ = sign;
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6556 if ((flags & F_ALT) && (c == 'x' || c == 'X')) {
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6557 assert(pbuf[0] == '0');
17450
f74f5aafdc7e Bug 415514 reported that e.g.
Tim Peters <tim.peters@gmail.com>
parents: 17425
diff changeset
6558 assert(pbuf[1] == c);
14713
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6559 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6560 *res++ = *pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6561 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats.
Tim Peters <tim.peters@gmail.com>
parents: 14492
diff changeset
6562 }
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6563 Py_UNICODE_COPY(res, pbuf, len);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6564 res += len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6565 rescnt -= len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6566 while (--width >= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6567 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6568 *res++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6569 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6570 if (dict && (argidx < arglen) && c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6571 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
6572 "not all arguments converted during string formatting");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6573 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6574 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6575 Py_XDECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6576 } /* '%' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6577 } /* until end */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6578 if (argidx < arglen && !dict) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6579 PyErr_SetString(PyExc_TypeError,
23494
76f376bea5e5 Patch 549187. Improve string formatting error message.
Raymond Hettinger <python@rcn.com>
parents: 23295
diff changeset
6580 "not all arguments converted during string formatting");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6581 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6582 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6583
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6584 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6585 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6586 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6587 Py_DECREF(uformat);
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6588 if (_PyUnicode_Resize(&result, reslen - rescnt))
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
6589 goto onError;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6590 return (PyObject *)result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6591
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6592 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6593 Py_XDECREF(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6594 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6595 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6596 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6597 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6598 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6599 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6600
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6601 static PyBufferProcs unicode_as_buffer = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6602 (getreadbufferproc) unicode_buffer_getreadbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6603 (getwritebufferproc) unicode_buffer_getwritebuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6604 (getsegcountproc) unicode_buffer_getsegcount,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6605 (getcharbufferproc) unicode_buffer_getcharbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6606 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6607
24280
1598a1e473e2 staticforward bites the dust.
Jeremy Hylton <jeremy@alum.mit.edu>
parents: 24071
diff changeset
6608 static PyObject *
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6609 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6610
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6611 static PyObject *
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6612 unicode_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6613 {
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6614 PyObject *x = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6615 static char *kwlist[] = {"string", "encoding", "errors", 0};
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6616 char *encoding = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6617 char *errors = NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6618
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6619 if (type != &PyUnicode_Type)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6620 return unicode_subtype_new(type, args, kwds);
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6621 if (!PyArg_ParseTupleAndKeywords(args, kwds, "|Oss:unicode",
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6622 kwlist, &x, &encoding, &errors))
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6623 return NULL;
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6624 if (x == NULL)
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6625 return (PyObject *)_PyUnicode_New(0);
20554
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
6626 if (encoding == NULL && errors == NULL)
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
6627 return PyObject_Unicode(x);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str()
Guido van Rossum <guido@python.org>
parents: 20277
diff changeset
6628 else
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6629 return PyUnicode_FromEncodedObject(x, encoding, errors);
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6630 }
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6631
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6632 static PyObject *
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6633 unicode_subtype_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6634 {
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6635 PyUnicodeObject *tmp, *pnew;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6636 int n;
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6637
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6638 assert(PyType_IsSubtype(type, &PyUnicode_Type));
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6639 tmp = (PyUnicodeObject *)unicode_new(&PyUnicode_Type, args, kwds);
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6640 if (tmp == NULL)
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6641 return NULL;
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6642 assert(PyUnicode_Check(tmp));
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6643 pnew = (PyUnicodeObject *) type->tp_alloc(type, n = tmp->length);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6644 if (pnew == NULL)
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6645 return NULL;
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6646 pnew->str = PyMem_NEW(Py_UNICODE, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6647 if (pnew->str == NULL) {
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6648 _Py_ForgetReference((PyObject *)pnew);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
6649 PyObject_Del(pnew);
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6650 return NULL;
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6651 }
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6652 Py_UNICODE_COPY(pnew->str, tmp->str, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6653 pnew->length = n;
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6654 pnew->hash = tmp->hash;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6655 Py_DECREF(tmp);
19886
3b0da13ace48 str_subtype_new, unicode_subtype_new:
Tim Peters <tim.peters@gmail.com>
parents: 19885
diff changeset
6656 return (PyObject *)pnew;
19593
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6657 }
a80b4b006eb7 Make unicode subclassable.
Guido van Rossum <guido@python.org>
parents: 19301
diff changeset
6658
23832
ed1ba37a46b7 Patch #568124: Add doc string macros.
Martin v. Löwis <martin@v.loewis.de>
parents: 23778
diff changeset
6659 PyDoc_STRVAR(unicode_doc,
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6660 "unicode(string [, encoding[, errors]]) -> object\n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6661 \n\
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6662 Create a new Unicode object from the given encoded string.\n\
24392
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
6663 encoding defaults to the current default string encoding.\n\
0a327b768868 tighten up the unicode object's docstring a tad
Skip Montanaro <skip@pobox.com>
parents: 24280
diff changeset
6664 errors can be 'strict', 'replace' or 'ignore' and defaults to 'strict'.");
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6665
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6666 PyTypeObject PyUnicode_Type = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6667 PyObject_HEAD_INIT(&PyType_Type)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6668 0, /* ob_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6669 "unicode", /* tp_name */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6670 sizeof(PyUnicodeObject), /* tp_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6671 0, /* tp_itemsize */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6672 /* Slots */
20277
2983b54256c7 Enable GC for new-style instances. This touches lots of files, since
Guido van Rossum <guido@python.org>
parents: 20018
diff changeset
6673 (destructor)unicode_dealloc, /* tp_dealloc */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6674 0, /* tp_print */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6675 0, /* tp_getattr */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6676 0, /* tp_setattr */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6677 (cmpfunc) unicode_compare, /* tp_compare */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6678 (reprfunc) unicode_repr, /* tp_repr */
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6679 &unicode_as_number, /* tp_as_number */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6680 &unicode_as_sequence, /* tp_as_sequence */
23778
245224d1b8c9 This is my nearly two year old patch
Michael W. Hudson <mwh@python.net>
parents: 23562
diff changeset
6681 &unicode_as_mapping, /* tp_as_mapping */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6682 (hashfunc) unicode_hash, /* tp_hash*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6683 0, /* tp_call*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6684 (reprfunc) unicode_str, /* tp_str */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6685 PyObject_GenericGetAttr, /* tp_getattro */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6686 0, /* tp_setattro */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6687 &unicode_as_buffer, /* tp_as_buffer */
26274
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6688 Py_TPFLAGS_DEFAULT | Py_TPFLAGS_CHECKTYPES |
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 26164
diff changeset
6689 Py_TPFLAGS_BASETYPE, /* tp_flags */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6690 unicode_doc, /* tp_doc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6691 0, /* tp_traverse */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6692 0, /* tp_clear */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6693 0, /* tp_richcompare */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6694 0, /* tp_weaklistoffset */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6695 0, /* tp_iter */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6696 0, /* tp_iternext */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6697 unicode_methods, /* tp_methods */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6698 0, /* tp_members */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6699 0, /* tp_getset */
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
6700 &PyBaseString_Type, /* tp_base */
18933
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6701 0, /* tp_dict */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6702 0, /* tp_descr_get */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6703 0, /* tp_descr_set */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6704 0, /* tp_dictoffset */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6705 0, /* tp_init */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6706 0, /* tp_alloc */
09df3254b49d Merge of descr-branch back into trunk.
Tim Peters <tim.peters@gmail.com>
parents: 18887
diff changeset
6707 unicode_new, /* tp_new */
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
6708 PyObject_Del, /* tp_free */
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6709 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6710
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6711 /* Initialize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6712
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
6713 void _PyUnicode_Init(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6714 {
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6715 int i;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6716
12645
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>:
Fred Drake <fdrake@acm.org>
parents: 12629
diff changeset
6717 /* Init the implementation */
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6718 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6719 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6720 unicode_empty = _PyUnicode_New(0);
12876
243593f11fcf Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12646
diff changeset
6721 strcpy(unicode_default_encoding, "ascii");
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6722 for (i = 0; i < 256; i++)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6723 unicode_latin1[i] = NULL;
23542
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
6724 if (PyType_Ready(&PyUnicode_Type) < 0)
220b98f06415 - A new type object, 'string', is added. This is a common base type
Guido van Rossum <guido@python.org>
parents: 23494
diff changeset
6725 Py_FatalError("Can't initialize 'unicode'");
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6726 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6727
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6728 /* Finalize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6729
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6730 void
13660
a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int,
Thomas Wouters <thomas@python.org>
parents: 13635
diff changeset
6731 _PyUnicode_Fini(void)
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6732 {
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
6733 PyUnicodeObject *u;
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6734 int i;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6735
14998
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
6736 Py_XDECREF(unicode_empty);
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free
Guido van Rossum <guido@python.org>
parents: 14869
diff changeset
6737 unicode_empty = NULL;
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
6738
17618
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6739 for (i = 0; i < 256; i++) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6740 if (unicode_latin1[i]) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6741 Py_DECREF(unicode_latin1[i]);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6742 unicode_latin1[i] = NULL;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6743 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6744 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a
Marc-André Lemburg <mal@egenix.com>
parents: 17600
diff changeset
6745
15001
a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u'
Barry Warsaw <barry@python.org>
parents: 14998
diff changeset
6746 for (u = unicode_freelist; u != NULL;) {
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6747 PyUnicodeObject *v = u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6748 u = *(PyUnicodeObject **)u;
12329
3cc77848a62a Marc-Andre Lemburg:
Guido van Rossum <guido@python.org>
parents: 12320
diff changeset
6749 if (v->str)
12576
1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring.
Guido van Rossum <guido@python.org>
parents: 12568
diff changeset
6750 PyMem_DEL(v->str);
13837
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in
Marc-André Lemburg <mal@egenix.com>
parents: 13828
diff changeset
6751 Py_XDECREF(v->defenc);
22981
dc1fb19d990d Remove PyMalloc_*.
Neil Schemenauer <nascheme@enme.ucalgary.ca>
parents: 22938
diff changeset
6752 PyObject_Del(v);
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6753 }
12899
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6754 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>:
Marc-André Lemburg <mal@egenix.com>
parents: 12876
diff changeset
6755 unicode_freelist_size = 0;
11968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by
Guido van Rossum <guido@python.org>
parents:
diff changeset
6756 }