cpython-lru_cache-weakref: Objects/unicodeobject.c annotate

annotate Objects/unicodeobject.c @ 30446:a8a1bef407eb 2.3

Backport Tim's portability improvement and comments.

author	Jeremy Hylton <jeremy@alum.mit.edu>
date	Wed, 17 Sep 2003 03:22:27 +0000
parents	784e38abed82
children	be0ca26ad574

rev	line source
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1 /*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3 Unicode implementation based on original code by Fredrik Lundh,
12646 17d1a3f1f085 Minimal change so I can add the rest of MAL's checkin message: Fred Drake <fdrake@acm.org> parents: 12645 diff changeset	4 modified by Marc-Andre Lemburg <mal@lemburg.com> according to the
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5 Unicode Integration Proposal (see file Misc/unicode.txt).
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6
13828 254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions. Guido van Rossum <guido@python.org> parents: 13761 diff changeset	7 Copyright (c) Corporation for National Research Initiatives.
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8
16029 55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	9 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	10 The original string type implementation is:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	11
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	12 Copyright (c) 1999 by Secret Labs AB
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	13 Copyright (c) 1999 by Fredrik Lundh
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	14
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	15 By obtaining, using, and/or copying this software and/or its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	16 associated documentation, you agree that you have read, understood,
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	17 and will comply with the following terms and conditions:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	18
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	19 Permission to use, copy, modify, and distribute this software and its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	20 associated documentation for any purpose and without fee is hereby
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	21 granted, provided that the above copyright notice appears in all
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	22 copies, and that both that copyright notice and this permission notice
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	23 appear in supporting documentation, and that the name of Secret Labs
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	24 AB or the author not be used in advertising or publicity pertaining to
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	25 distribution of the software without specific, written prior
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	26 permission.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	27
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	28 SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	29 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	30 FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	31 ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	32 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	33 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	34 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	35 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	36
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	37 */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	38
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	39 #include "Python.h"
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	40
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	41 #include "unicodeobject.h"
13209 b11e0a898de0 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13196 diff changeset	42 #include "ucnhash.h"
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	43
24071 8142f8d668a0 Patch #569753: Remove support for WIN16. Martin v. Löwis <martin@v.loewis.de> parents: 23837 diff changeset	44 #ifdef MS_WINDOWS
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	45 #include <windows.h>
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	46 #endif
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	47
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	48 /* Limit for the Unicode object free list */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	49
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	50 #define MAX_UNICODE_FREELIST_SIZE 1024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	51
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	52 /* Limit for the Unicode object free list stay alive optimization.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	53
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	54 The implementation will keep allocated Unicode memory intact for
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	55 all objects on the free list having a size less than this
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	56 limit. This reduces malloc() overhead for small Unicode objects.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	57
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	58 At worst this will result in MAX_UNICODE_FREELIST_SIZE *
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	59 (sizeof(PyUnicodeObject) + KEEPALIVE_SIZE_LIMIT +
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	60 malloc()-overhead) bytes of unused garbage.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	61
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	62 Setting the limit to 0 effectively turns the feature off.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	63
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	64 Note: This is an experimental feature ! If you get core dumps when
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	65 using Unicode objects, turn this feature off.
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	66
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	67 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	68
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	69 #define KEEPALIVE_SIZE_LIMIT 9
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	70
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	71 /* Endianness switches; defaults to little endian */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	72
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	73 #ifdef WORDS_BIGENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	74 # define BYTEORDER_IS_BIG_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	75 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	76 # define BYTEORDER_IS_LITTLE_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	77 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	78
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	79 /* --- Globals ------------------------------------------------------------
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	80
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	81 The globals are initialized by the _PyUnicode_Init() API and should
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	82 not be used before calling that API.
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	83
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	84 */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	85
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	86 /* Free list for Unicode objects */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	87 static PyUnicodeObject *unicode_freelist;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	88 static int unicode_freelist_size;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	89
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	90 /* The empty Unicode object is shared to improve performance. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	91 static PyUnicodeObject *unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	92
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	93 /* Single character Unicode strings in the Latin-1 range are being
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	94 shared as well. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	95 static PyUnicodeObject *unicode_latin1[256];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	96
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	97 /* Default encoding to use and assume when NULL is passed as encoding
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	98 parameter; it is initialized by _PyUnicode_Init().
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	99
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	100 Always use the PyUnicode_SetDefaultEncoding() and
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	101 PyUnicode_GetDefaultEncoding() APIs to access this global.
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	102
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	103 */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	104 static char unicode_default_encoding[100];
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	105
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	106 Py_UNICODE
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	107 PyUnicode_GetMax(void)
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	108 {
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	109 #ifdef Py_UNICODE_WIDE
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	110 return 0x10FFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	111 #else
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	112 /* This is actually an illegal character, so it should
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	113 not be passed to unichr. */
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	114 return 0xFFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	115 #endif
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	116 }
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	117
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	118 /* --- Unicode Object ----------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	119
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	120 static
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	121 int unicode_resize(register PyUnicodeObject *unicode,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	122 int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	123 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	124 void *oldstr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	125
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	126 /* Shortcut if there's nothing much to do. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	127 if (unicode->length == length)
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	128 goto reset;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	129
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	130 /* Resizing shared object (unicode_empty or single character
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	131 objects) in-place is not allowed. Use PyUnicode_Resize()
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	132 instead ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	133 if (unicode == unicode_empty \|\|
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	134 (unicode->length == 1 &&
30446 a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	135 /* MvL said unicode->str[] may be signed. Python generally assumes
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	136 * an int contains at least 32 bits, and we don't use more than
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	137 * 32 bits even in a UCS4 build, so casting to unsigned int should
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	138 * be correct.
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	139 */
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	140 (unsigned int)unicode->str[0] < 256U &&
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	141 unicode_latin1[unicode->str[0]] == unicode)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	142 PyErr_SetString(PyExc_SystemError,
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	143 "can't resize shared unicode objects");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	144 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	145 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	146
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	147 /* We allocate one more byte to make sure the string is
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	148 Ux0000 terminated -- XXX is this needed ? */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	149 oldstr = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	150 PyMem_RESIZE(unicode->str, Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	151 if (!unicode->str) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	152 unicode->str = oldstr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	153 PyErr_NoMemory();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	154 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	155 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	156 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	157 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	158
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	159 reset:
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	160 /* Reset the object caches */
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	161 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	162 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	163 unicode->defenc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	164 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	165 unicode->hash = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	166
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	167 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	168 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	169
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	170 /* We allocate one more byte to make sure the string is
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	171 Ux0000 terminated -- XXX is this needed ?
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	172
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	173 XXX This allocator could further be enhanced by assuring that the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	174 free list never reduces its size below 1.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	175
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	176 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	177
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	178 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	179 PyUnicodeObject *_PyUnicode_New(int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	180 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	181 register PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	182
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	183 /* Optimization for empty strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	184 if (length == 0 && unicode_empty != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	185 Py_INCREF(unicode_empty);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	186 return unicode_empty;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	187 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	188
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	189 /* Unicode freelist & memory allocation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	190 if (unicode_freelist) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	191 unicode = unicode_freelist;
12939 88004f25dc25 Vladimir MARANGOZOV <Vladimir.Marangozov@inrialpes.fr>: Marc-André Lemburg <mal@egenix.com> parents: 12924 diff changeset	192 unicode_freelist = (PyUnicodeObject *)unicode;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	193 unicode_freelist_size--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	194 if (unicode->str) {
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	195 /* Keep-Alive optimization: we only upsize the buffer,
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	196 never downsize it. */
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	197 if ((unicode->length < length) &&
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	198 unicode_resize(unicode, length)) {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	199 PyMem_DEL(unicode->str);
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	200 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	202 }
18120 c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	203 else {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	204 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
18120 c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	205 }
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	206 PyObject_INIT(unicode, &PyUnicode_Type);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	207 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	208 else {
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	209 unicode = PyObject_New(PyUnicodeObject, &PyUnicode_Type);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	210 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	211 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	212 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	213 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	214
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	215 if (!unicode->str) {
8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	216 PyErr_NoMemory();
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	217 goto onError;
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	218 }
30446 a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	219 /* Initialize the first element to guard against cases where
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	220 * the caller fails before initializing str -- unicode_resize()
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	221 * reads str[0], and the Keep-Alive optimization can keep memory
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	222 * allocated for str alive across a call to unicode_dealloc(unicode).
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	223 * We don't want unicode_resize to read uninitialized memory in
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	224 * that case.
a8a1bef407eb Backport Tim's portability improvement and comments. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30440 diff changeset	225 */
30440 784e38abed82 Backport: Double-fix of crash in Unicode freelist handling. Jeremy Hylton <jeremy@alum.mit.edu> parents: 30217 diff changeset	226 unicode->str[0] = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	227 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	228 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	229 unicode->hash = -1;
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	230 unicode->defenc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	231 return unicode;
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	232
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	233 onError:
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	234 _Py_ForgetReference((PyObject *)unicode);
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	235 PyObject_Del(unicode);
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	236 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	237 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	239 static
20277 2983b54256c7 Enable GC for new-style instances. This touches lots of files, since Guido van Rossum <guido@python.org> parents: 20018 diff changeset	240 void unicode_dealloc(register PyUnicodeObject *unicode)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	241 {
21262 e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode). Guido van Rossum <guido@python.org> parents: 21082 diff changeset	242 if (PyUnicode_CheckExact(unicode) &&
e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode). Guido van Rossum <guido@python.org> parents: 21082 diff changeset	243 unicode_freelist_size < MAX_UNICODE_FREELIST_SIZE) {
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	244 /* Keep-Alive optimization */
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	245 if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	246 PyMem_DEL(unicode->str);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	247 unicode->str = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	248 unicode->length = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	249 }
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	250 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	251 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	252 unicode->defenc = NULL;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	253 }
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	254 /* Add to free list */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	255 (PyUnicodeObject *)unicode = unicode_freelist;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	256 unicode_freelist = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	257 unicode_freelist_size++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	258 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	259 else {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	260 PyMem_DEL(unicode->str);
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	261 Py_XDECREF(unicode->defenc);
21262 e8fea95a82a4 Fix for #489669 (Neil Norwitz): memory leak in test_descr (unicode). Guido van Rossum <guido@python.org> parents: 21082 diff changeset	262 unicode->ob_type->tp_free((PyObject *)unicode);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	264 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	265
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	266 int PyUnicode_Resize(PyObject **unicode,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	267 int length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	268 {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	269 register PyUnicodeObject *v;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	270
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	271 /* Argument checks */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	272 if (unicode == NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	273 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	274 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	275 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	276 v = (PyUnicodeObject )unicode;
25719 f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	277 if (v == NULL \|\| !PyUnicode_Check(v) \|\| v->ob_refcnt != 1 \|\| length < 0) {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	278 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	279 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	280 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	281
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	282 /* Resizing unicode_empty and single character objects is not
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	283 possible since these are being shared. We simply return a fresh
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	284 copy with the same Unicode content. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	285 if (v->length != length &&
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	286 (v == unicode_empty \|\| v->length == 1)) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	287 PyUnicodeObject *w = _PyUnicode_New(length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	288 if (w == NULL)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	289 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	290 Py_UNICODE_COPY(w->str, v->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	291 length < v->length ? length : v->length);
28242 fe1d571c3f80 Sf patch #700047: unicode object leaks refcount on resizing Raymond Hettinger <python@rcn.com> parents: 27826 diff changeset	292 Py_DECREF(*unicode);
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	293 unicode = (PyObject )w;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	294 return 0;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	295 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	296
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	297 /* Note that we don't have to modify *unicode for unshared Unicode
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	298 objects, since we can modify them in-place. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	299 return unicode_resize(v, length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	300 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	301
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	302 /* Internal API for use in unicodeobject.c only ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	303 #define _PyUnicode_Resize(unicodevar, length) \
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	304 PyUnicode_Resize(((PyObject **)(unicodevar)), length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	305
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	306 PyObject PyUnicode_FromUnicode(const Py_UNICODE u,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	307 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	308 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	309 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	310
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	311 /* If the Unicode data is known at construction time, we can apply
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	312 some optimizations which share commonly used objects. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	313 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	314
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	315 /* Optimization for empty strings */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	316 if (size == 0 && unicode_empty != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	317 Py_INCREF(unicode_empty);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	318 return (PyObject *)unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	319 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	320
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	321 /* Single character Unicode objects in the Latin-1 range are
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	322 shared when using this constructor */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	323 if (size == 1 && *u < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	324 unicode = unicode_latin1[*u];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	325 if (!unicode) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	326 unicode = _PyUnicode_New(1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	327 if (!unicode)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	328 return NULL;
18051 e897550b4748 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode. Marc-André Lemburg <mal@egenix.com> parents: 17995 diff changeset	329 unicode->str[0] = *u;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	330 unicode_latin1[*u] = unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	331 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	332 Py_INCREF(unicode);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	333 return (PyObject *)unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	334 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	335 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	336
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	337 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	338 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	339 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	340
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	341 /* Copy the Unicode data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	342 if (u != NULL)
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	343 Py_UNICODE_COPY(unicode->str, u, size);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	344
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	345 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	346 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	347
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	348 #ifdef HAVE_WCHAR_H
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	349
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	350 PyObject PyUnicode_FromWideChar(register const wchar_t w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	351 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	352 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	353 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	354
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	355 if (w == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	356 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	357 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	358 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	359
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	360 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	361 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	362 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	363
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	364 /* Copy the wchar_t data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	365 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	366 memcpy(unicode->str, w, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	367 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	368 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	369 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	370 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	371 u = PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	372 for (i = size; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	373 u++ = w++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	374 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	375 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	376
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	377 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	378 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	379
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	380 int PyUnicode_AsWideChar(PyUnicodeObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	381 register wchar_t *w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	382 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	383 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	384 if (unicode == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	385 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	386 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	387 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	388 if (size > PyUnicode_GET_SIZE(unicode))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	389 size = PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	390 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	391 memcpy(w, unicode->str, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	392 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	393 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	394 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	395 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	396 u = PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	397 for (i = size; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	398 w++ = u++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	399 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	400 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	401
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	402 return size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	403 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	404
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	405 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	406
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	407 PyObject *PyUnicode_FromOrdinal(int ordinal)
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	408 {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	409 Py_UNICODE s[2];
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	410
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	411 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	412 if (ordinal < 0 \|\| ordinal > 0x10ffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	413 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	414 "unichr() arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	415 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	416 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	417 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	418 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	419 if (ordinal < 0 \|\| ordinal > 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	420 PyErr_SetString(PyExc_ValueError,
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	421 "unichr() arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	422 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	423 return NULL;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	424 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	425 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	426
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	427 if (ordinal <= 0xffff) {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	428 /* UCS-2 character */
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	429 s[0] = (Py_UNICODE) ordinal;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	430 return PyUnicode_FromUnicode(s, 1);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	431 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	432 else {
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	433 #ifndef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	434 /* UCS-4 character. store as two surrogate characters */
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	435 ordinal -= 0x10000L;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	436 s[0] = 0xD800 + (Py_UNICODE) (ordinal >> 10);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	437 s[1] = 0xDC00 + (Py_UNICODE) (ordinal & 0x03FF);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	438 return PyUnicode_FromUnicode(s, 2);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	439 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	440 s[0] = (Py_UNICODE)ordinal;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	441 return PyUnicode_FromUnicode(s, 1);
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	442 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	443 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	444 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	445
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	446 PyObject PyUnicode_FromObject(register PyObject obj)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	447 {
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	448 /* XXX Perhaps we should make this API an alias of
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	449 PyObject_Unicode() instead ?! */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	450 if (PyUnicode_CheckExact(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	451 Py_INCREF(obj);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	452 return obj;
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	453 }
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	454 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	455 /* For a Unicode subtype that's not a Unicode object,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	456 return a true Unicode object with the same data. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	457 return PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(obj),
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	458 PyUnicode_GET_SIZE(obj));
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	459 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	460 return PyUnicode_FromEncodedObject(obj, NULL, "strict");
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	461 }
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	462
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	463 PyObject PyUnicode_FromEncodedObject(register PyObject obj,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	464 const char *encoding,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	465 const char *errors)
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	466 {
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	467 const char *s = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	468 int len;
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	469 PyObject *v;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	470
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	471 if (obj == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	472 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	473 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	474 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	475
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	476 #if 0
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	477 /* For b/w compatibility we also accept Unicode objects provided
20563 9058e49c6c57 Additional test and documentation for the unicode() changes. Marc-André Lemburg <mal@egenix.com> parents: 20554 diff changeset	478 that no encodings is given and then redirect to
9058e49c6c57 Additional test and documentation for the unicode() changes. Marc-André Lemburg <mal@egenix.com> parents: 20554 diff changeset	479 PyObject_Unicode() which then applies the additional logic for
9058e49c6c57 Additional test and documentation for the unicode() changes. Marc-André Lemburg <mal@egenix.com> parents: 20554 diff changeset	480 Unicode subclasses.
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	481
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	482 NOTE: This API should really only be used for object which
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	483 represent encoded Unicode !
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	484
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	485 */
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	486 if (PyUnicode_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	487 if (encoding) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	488 PyErr_SetString(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	489 "decoding Unicode is not supported");
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	490 return NULL;
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	491 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	492 return PyObject_Unicode(obj);
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	493 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	494 #else
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	495 if (PyUnicode_Check(obj)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	496 PyErr_SetString(PyExc_TypeError,
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	497 "decoding Unicode is not supported");
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	498 return NULL;
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	499 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	500 #endif
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	501
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	502 /* Coerce object */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	503 if (PyString_Check(obj)) {
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	504 s = PyString_AS_STRING(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	505 len = PyString_GET_SIZE(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	506 }
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	507 else if (PyObject_AsCharBuffer(obj, &s, &len)) {
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	508 /* Overwrite the error message with something more useful in
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	509 case of a TypeError. */
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	510 if (PyErr_ExceptionMatches(PyExc_TypeError))
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	511 PyErr_Format(PyExc_TypeError,
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	512 "coercing to Unicode: need string or buffer, "
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	513 "%.80s found",
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	514 obj->ob_type->tp_name);
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	515 goto onError;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	516 }
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	517
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	518 /* Convert to Unicode */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	519 if (len == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	520 Py_INCREF(unicode_empty);
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	521 v = (PyObject *)unicode_empty;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	522 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	523 else
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	524 v = PyUnicode_Decode(s, len, encoding, errors);
15952 1f92d63b20f0 This patch adds a new builtin unistr() which behaves like str() Marc-André Lemburg <mal@egenix.com> parents: 15910 diff changeset	525
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	526 return v;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	527
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	528 onError:
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	529 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	530 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	531
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	532 PyObject PyUnicode_Decode(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	533 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	534 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	535 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	536 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	537 PyObject buffer = NULL, unicode;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	538
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	539 if (encoding == NULL)
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	540 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	541
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	542 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	543 if (strcmp(encoding, "utf-8") == 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	544 return PyUnicode_DecodeUTF8(s, size, errors);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	545 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	546 return PyUnicode_DecodeLatin1(s, size, errors);
29706 a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	547 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	548 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	549 return PyUnicode_DecodeMBCS(s, size, errors);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	550 #endif
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	551 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	552 return PyUnicode_DecodeASCII(s, size, errors);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	553
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	554 /* Decode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	555 buffer = PyBuffer_FromMemory((void *)s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	556 if (buffer == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	557 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	558 unicode = PyCodec_Decode(buffer, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	559 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	560 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	561 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	562 PyErr_Format(PyExc_TypeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	563 "decoder did not return an unicode object (type=%.400s)",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	564 unicode->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	565 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	566 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	567 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	568 Py_DECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	569 return unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	570
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	571 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	572 Py_XDECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	573 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	574 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	575
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	576 PyObject PyUnicode_Encode(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	577 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	578 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	579 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	580 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	581 PyObject v, unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	582
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	583 unicode = PyUnicode_FromUnicode(s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	584 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	585 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	586 v = PyUnicode_AsEncodedString(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	587 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	588 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	589 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	590
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	591 PyObject PyUnicode_AsEncodedString(PyObject unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	592 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	593 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	594 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	595 PyObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	596
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	597 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	598 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	599 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	600 }
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	601
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	602 if (encoding == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	603 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	604
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	605 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	606 if (errors == NULL) {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	607 if (strcmp(encoding, "utf-8") == 0)
17995 837825a779c1 fix bogus indentation Jeremy Hylton <jeremy@alum.mit.edu> parents: 17887 diff changeset	608 return PyUnicode_AsUTF8String(unicode);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	609 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	610 return PyUnicode_AsLatin1String(unicode);
29706 a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	611 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	612 else if (strcmp(encoding, "mbcs") == 0)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	613 return PyUnicode_AsMBCSString(unicode);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	614 #endif
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	615 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	616 return PyUnicode_AsASCIIString(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	617 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	618
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	619 /* Encode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	620 v = PyCodec_Encode(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	621 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	622 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	623 /* XXX Should we really enforce this ? */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	624 if (!PyString_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	625 PyErr_Format(PyExc_TypeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	626 "encoder did not return a string object (type=%.400s)",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	627 v->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	628 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	629 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	630 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	631 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	632
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	633 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	634 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	635 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	636
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	637 PyObject _PyUnicode_AsDefaultEncodedString(PyObject unicode,
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	638 const char *errors)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	639 {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	640 PyObject v = ((PyUnicodeObject )unicode)->defenc;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	641
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	642 if (v)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	643 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	644 v = PyUnicode_AsEncodedString(unicode, NULL, errors);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	645 if (v && errors == NULL)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	646 ((PyUnicodeObject *)unicode)->defenc = v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	647 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	648 }
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	649
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	650 Py_UNICODE PyUnicode_AsUnicode(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	651 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	652 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	653 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	654 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	655 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	656 return PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	657
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	658 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	659 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	660 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	661
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	662 int PyUnicode_GetSize(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	663 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	664 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	665 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	666 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	667 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	668 return PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	669
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	670 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	671 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	672 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	673
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	674 const char *PyUnicode_GetDefaultEncoding(void)
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	675 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	676 return unicode_default_encoding;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	677 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	678
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	679 int PyUnicode_SetDefaultEncoding(const char *encoding)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	680 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	681 PyObject *v;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	682
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	683 /* Make sure the encoding is valid. As side effect, this also
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	684 loads the encoding into the codec registry cache. */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	685 v = _PyCodec_Lookup(encoding);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	686 if (v == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	687 goto onError;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	688 Py_DECREF(v);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	689 strncpy(unicode_default_encoding,
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	690 encoding,
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	691 sizeof(unicode_default_encoding));
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	692 return 0;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	693
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	694 onError:
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	695 return -1;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	696 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	697
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	698 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	699 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	700 if no exception occured, copy the replacement to the output
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	701 and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	702 return 0 on success, -1 on error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	703 */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	704
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	705 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	706 int unicode_decode_call_errorhandler(const char errors, PyObject *errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	707 const char encoding, const char reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	708 const char input, int insize, int startinpos, int endinpos, PyObject exceptionObject, const char *inptr,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	709 PyObject *output, int outpos, Py_UNICODE **outptr)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	710 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	711 static char *argparse = "O!i;decoding error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	712
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	713 PyObject *restuple = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	714 PyObject *repunicode = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	715 int outsize = PyUnicode_GET_SIZE(*output);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	716 int requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	717 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	718 Py_UNICODE *repptr;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	719 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	720 int res = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	721
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	722 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	723 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	724 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	725 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	726 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	727
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	728 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	729 *exceptionObject = PyUnicodeDecodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	730 encoding, input, insize, startinpos, endinpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	731 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	732 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	733 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	734 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	735 if (PyUnicodeDecodeError_SetStart(exceptionObject, startinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	736 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	737 if (PyUnicodeDecodeError_SetEnd(exceptionObject, endinpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	738 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	739 if (PyUnicodeDecodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	740 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	741 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	742
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	743 restuple = PyObject_CallFunctionObjArgs(errorHandler, exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	744 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	745 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	746 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	747 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	748 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	749 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	750 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type, &repunicode, &newpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	751 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	752 if (newpos<0)
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	753 newpos = insize+newpos;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	754 if (newpos<0 \|\| newpos>insize) {
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	755 PyErr_Format(PyExc_IndexError, "position %d from error handler out of bounds", newpos);
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	756 goto onError;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	757 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	758
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	759 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	760 have+the replacement+the rest of the string (starting
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	761 at the new input position), so we won't have to check space
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	762 when there are no errors in the rest of the string) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	763 repptr = PyUnicode_AS_UNICODE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	764 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	765 requiredsize = *outpos + repsize + insize-newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	766 if (requiredsize > outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	767 if (requiredsize<2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	768 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	769 if (PyUnicode_Resize(output, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	770 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	771 outptr = PyUnicode_AS_UNICODE(output) + *outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	772 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	773 *endinpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	774 *inptr = input + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	775 Py_UNICODE_COPY(*outptr, repptr, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	776 *outptr += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	777 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	778 /* we made it! */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	779 res = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	780
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	781 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	782 Py_XDECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	783 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	784 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	785
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	786 /* --- UTF-7 Codec -------------------------------------------------------- */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	787
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	788 /* see RFC2152 for details */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	789
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	790 static
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	791 char utf7_special[128] = {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	792 /* indicate whether a UTF-7 character is special i.e. cannot be directly
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	793 encoded:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	794 0 - not special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	795 1 - special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	796 2 - whitespace (optional)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	797 3 - RFC2152 Set O (optional) */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	798 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	799 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	800 2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	801 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	802 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	803 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	804 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	805 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	806
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	807 };
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	808
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	809 #define SPECIAL(c, encodeO, encodeWS) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	810 (((c)>127 \|\| utf7_special[(c)] == 1) \|\| \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	811 (encodeWS && (utf7_special[(c)] == 2)) \|\| \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	812 (encodeO && (utf7_special[(c)] == 3)))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	813
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	814 #define B64(n) ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[(n) & 0x3f])
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	815 #define B64CHAR(c) (isalnum(c) \|\| (c) == '+' \|\| (c) == '/')
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	816 #define UB64(c) ((c) == '+' ? 62 : (c) == '/' ? 63 : (c) >= 'a' ? \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	817 (c) - 71 : (c) >= 'A' ? (c) - 65 : (c) + 4)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	818
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	819 #define ENCODE(out, ch, bits) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	820 while (bits >= 6) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	821 *out++ = B64(ch >> (bits-6)); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	822 bits -= 6; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	823 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	824
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	825 #define DECODE(out, ch, bits, surrogate) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	826 while (bits >= 16) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	827 Py_UNICODE outCh = (Py_UNICODE) ((ch >> (bits-16)) & 0xffff); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	828 bits -= 16; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	829 if (surrogate) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	830 /* We have already generated an error for the high surrogate
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	831 so let's not bother seeing if the low surrogate is correct or not */\
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	832 surrogate = 0; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	833 } else if (0xDC00 <= outCh && outCh <= 0xDFFF) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	834 /* This is a surrogate pair. Unfortunately we can't represent \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	835 it in a 16-bit character */ \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	836 surrogate = 1; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	837 errmsg = "code pairs are not supported"; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	838 goto utf7Error; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	839 } else { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	840 *out++ = outCh; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	841 } \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	842 } \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	843
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	844 PyObject PyUnicode_DecodeUTF7(const char s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	845 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	846 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	847 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	848 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	849 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	850 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	851 int outpos;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	852 const char *e;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	853 PyUnicodeObject *unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	854 Py_UNICODE *p;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	855 const char *errmsg = "";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	856 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	857 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	858 unsigned long charsleft = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	859 int surrogate = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	860 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	861 PyObject *exc = NULL;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	862
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	863 unicode = _PyUnicode_New(size);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	864 if (!unicode)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	865 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	866 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	867 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	868
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	869 p = unicode->str;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	870 e = s + size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	871
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	872 while (s < e) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	873 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	874 restart:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	875 ch = *s;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	876
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	877 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	878 if ((ch == '-') \|\| !B64CHAR(ch)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	879 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	880 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	881
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	882 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	883 if (bitsleft >= 6) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	884 /* The shift sequence has a partial character in it. If
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	885 bitsleft < 6 then we could just classify it as padding
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	886 but that is not the case here */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	887
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	888 errmsg = "partial character in shift sequence";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	889 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	890 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	891 /* According to RFC2152 the remaining bits should be zero. We
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	892 choose to signal an error/insert a replacement character
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	893 here so indicate the potential of a misencoded character. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	894
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	895 /* On x86, a << b == a << (b%32) so make sure that bitsleft != 0 */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	896 if (bitsleft && charsleft << (sizeof(charsleft) * 8 - bitsleft)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	897 errmsg = "non-zero padding bits in shift sequence";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	898 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	899 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	900
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	901 if (ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	902 if ((s < e) && (*(s) == '-')) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	903 *p++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	904 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	905 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	906 } else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	907 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	908 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	909 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	910 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	911 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	912 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	913 charsleft = (charsleft << 6) \| UB64(ch);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	914 bitsleft += 6;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	915 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	916 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	917 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	918 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	919 else if ( ch == '+' ) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	920 startinpos = s-starts;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	921 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	922 if (s < e && *s == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	923 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	924 *p++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	925 } else
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	926 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	927 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	928 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	929 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	930 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	931 else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	932 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	933 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	934 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	935 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	936 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	937 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	938 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	939 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	940 continue;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	941 utf7Error:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	942 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	943 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	944 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	945 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	946 "utf7", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	947 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	948 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	949 goto onError;
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	950 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	951
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	952 if (inShift) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	953 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	954 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	955 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	956 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	957 "utf7", "unterminated shift sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	958 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	959 (PyObject **)&unicode, &outpos, &p))
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	960 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	961 if (s < e)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	962 goto restart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	963 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	964
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	965 if (_PyUnicode_Resize(&unicode, p - PyUnicode_AS_UNICODE(unicode)))
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	966 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	967
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	968 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	969 Py_XDECREF(exc);
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	970 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	971
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	972 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	973 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	974 Py_XDECREF(exc);
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	975 Py_DECREF(unicode);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	976 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	977 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	978
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	979
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	980 PyObject PyUnicode_EncodeUTF7(const Py_UNICODE s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	981 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	982 int encodeSetO,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	983 int encodeWhiteSpace,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	984 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	985 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	986 PyObject *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	987 /* It might be possible to tighten this worst case */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	988 unsigned int cbAllocated = 5 * size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	989 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	990 int i = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	991 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	992 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	993 char * out;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	994 char * start;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	995
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	996 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	997 return PyString_FromStringAndSize(NULL, 0);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	998
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	999 v = PyString_FromStringAndSize(NULL, cbAllocated);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1000 if (v == NULL)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1001 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1002
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1003 start = out = PyString_AS_STRING(v);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1004 for (;i < size; ++i) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1005 Py_UNICODE ch = s[i];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1006
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1007 if (!inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1008 if (ch == '+') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1009 *out++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1010 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1011 } else if (SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1012 charsleft = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1013 bitsleft = 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1014 *out++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1015 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1016 inShift = bitsleft > 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1017 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1018 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1019 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1020 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1021 if (!SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1022 *out++ = B64(charsleft << (6-bitsleft));
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1023 charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1024 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1025 /* Characters not in the BASE64 set implicitly unshift the sequence
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1026 so no '-' is required, except if the character is itself a '-' */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1027 if (B64CHAR(ch) \|\| ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1028 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1029 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1030 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1031 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1032 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1033 bitsleft += 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1034 charsleft = (charsleft << 16) \| ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1035 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1036
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1037 /* If the next character is special then we dont' need to terminate
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1038 the shift sequence. If the next character is not a BASE64 character
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1039 or '-' then the shift sequence will be terminated implicitly and we
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1040 don't have to insert a '-'. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1041
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1042 if (bitsleft == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1043 if (i + 1 < size) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1044 Py_UNICODE ch2 = s[i+1];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1045
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1046 if (SPECIAL(ch2, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1047
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1048 } else if (B64CHAR(ch2) \|\| ch2 == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1049 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1050 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1051 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1052 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1053 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1054
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1055 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1056 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1057 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1058 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1059 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1060 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1061 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1062 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1063 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1064 if (bitsleft) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1065 *out++= B64(charsleft << (6-bitsleft) );
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1066 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1067 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1068
23295 6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people Tim Peters <tim.peters@gmail.com> parents: 23294 diff changeset	1069 _PyString_Resize(&v, out - start);
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1070 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1071 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1072
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1073 #undef SPECIAL
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1074 #undef B64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1075 #undef B64CHAR
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1076 #undef UB64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1077 #undef ENCODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1078 #undef DECODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	1079
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1080 /* --- UTF-8 Codec -------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1081
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1082 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1083 char utf8_code_length[256] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1084 /* Map UTF-8 encoded prefix byte to sequence length. zero means
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1085 illegal prefix. see RFC 2279 for details */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1086 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1087 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1088 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1089 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1090 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1091 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1092 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1093 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1094 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1095 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1096 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1097 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1098 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1099 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1100 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1101 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1102 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1103
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1104 PyObject PyUnicode_DecodeUTF8(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1105 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1106 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1107 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1108 const char *starts = s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1109 int n;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1110 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1111 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1112 int outpos;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1113 const char *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1114 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1115 Py_UNICODE *p;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1116 const char *errmsg = "";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1117 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1118 PyObject *exc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1119
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1120 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1121 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1122 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1123 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1124 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1125 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1126 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1127
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1128 /* Unpack UTF-8 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1129 p = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1130 e = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1131
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1132 while (s < e) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1133 Py_UCS4 ch = (unsigned char)*s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1134
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1135 if (ch < 0x80) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1136 *p++ = (Py_UNICODE)ch;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1137 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1138 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1139 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1140
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1141 n = utf8_code_length[ch];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1142
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1143 if (s + n > e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1144 errmsg = "unexpected end of data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1145 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1146 endinpos = size;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1147 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1148 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1149
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1150 switch (n) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1151
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1152 case 0:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1153 errmsg = "unexpected code byte";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1154 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1155 endinpos = startinpos+1;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1156 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1157
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1158 case 1:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1159 errmsg = "internal error";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1160 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1161 endinpos = startinpos+1;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1162 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1163
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1164 case 2:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1165 if ((s[1] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1166 errmsg = "invalid data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1167 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1168 endinpos = startinpos+2;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1169 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1170 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1171 ch = ((s[0] & 0x1f) << 6) + (s[1] & 0x3f);
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1172 if (ch < 0x80) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1173 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1174 endinpos = startinpos+2;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1175 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1176 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1177 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1178 else
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1179 *p++ = (Py_UNICODE)ch;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1180 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1181
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1182 case 3:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1183 if ((s[1] & 0xc0) != 0x80 \|\|
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1184 (s[2] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1185 errmsg = "invalid data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1186 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1187 endinpos = startinpos+3;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1188 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1189 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1190 ch = ((s[0] & 0x0f) << 12) + ((s[1] & 0x3f) << 6) + (s[2] & 0x3f);
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1191 if (ch < 0x0800) {
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1192 /* Note: UTF-8 encodings of surrogates are considered
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1193 legal UTF-8 sequences;
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1194
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1195 XXX For wide builds (UCS-4) we should probably try
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1196 to recombine the surrogates into a single code
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1197 unit.
abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1198 */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1199 errmsg = "illegal encoding";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1200 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1201 endinpos = startinpos+3;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1202 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1203 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1204 else
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1205 *p++ = (Py_UNICODE)ch;
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1206 break;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1207
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1208 case 4:
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1209 if ((s[1] & 0xc0) != 0x80 \|\|
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1210 (s[2] & 0xc0) != 0x80 \|\|
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1211 (s[3] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1212 errmsg = "invalid data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1213 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1214 endinpos = startinpos+4;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1215 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1216 }
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1217 ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1218 ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1219 /* validate and convert to UTF-16 */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1220 if ((ch < 0x10000) /* minimum value allowed for 4
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1221 byte encoding */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1222 \|\| (ch > 0x10ffff)) /* maximum value allowed for
21886 abedb2418f62 Fix to the UTF-8 encoder: it failed on 0-length input strings. Marc-André Lemburg <mal@egenix.com> parents: 21884 diff changeset	1223 UTF-16 */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1224 {
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1225 errmsg = "illegal encoding";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1226 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1227 endinpos = startinpos+4;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1228 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1229 }
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1230 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1231 *p++ = (Py_UNICODE)ch;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1232 #else
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1233 /* compute and append the two surrogates: */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1234
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1235 /* translate from 10000..10FFFF to 0..FFFF */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1236 ch -= 0x10000;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1237
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1238 /* high surrogate = top 10 bits added to D800 */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1239 *p++ = (Py_UNICODE)(0xD800 + (ch >> 10));
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1240
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1241 /* low surrogate = bottom 10 bits added to DC00 */
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	1242 *p++ = (Py_UNICODE)(0xDC00 + (ch & 0x03FF));
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1243 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1244 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1245
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1246 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1247 /* Other sizes are only needed for UCS-4 */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1248 errmsg = "unsupported Unicode code range";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1249 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1250 endinpos = startinpos+n;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1251 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1252 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1253 s += n;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1254 continue;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1255
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1256 utf8Error:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1257 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1258 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1259 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1260 "utf8", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1261 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1262 (PyObject **)&unicode, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1263 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1264 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1265
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1266 /* Adjust length */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	1267 if (_PyUnicode_Resize(&unicode, p - unicode->str))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1268 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1269
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1270 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1271 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1272 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1273
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1274 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1275 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1276 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1277 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1278 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1279 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1280
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1281 /* Allocation strategy: if the string is short, convert into a stack buffer
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1282 and allocate exactly as much space needed at the end. Else allocate the
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1283 maximum possible needed (4 result bytes per Unicode character), and return
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1284 the excess memory at the end.
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1285 */
23198 950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two Tim Peters <tim.peters@gmail.com> parents: 23192 diff changeset	1286 PyObject *
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two Tim Peters <tim.peters@gmail.com> parents: 23192 diff changeset	1287 PyUnicode_EncodeUTF8(const Py_UNICODE *s,
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two Tim Peters <tim.peters@gmail.com> parents: 23192 diff changeset	1288 int size,
950a22eb8cc8 PyUnicode_EncodeUTF8: squash compiler wng. The difference of two Tim Peters <tim.peters@gmail.com> parents: 23192 diff changeset	1289 const char *errors)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1290 {
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1291 #define MAX_SHORT_UNICHARS 300 /* largest size we'll do on the stack */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1292
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1293 int i; /* index into s of next input byte */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1294 PyObject v; / result string object */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1295 char p; / next free byte in output buffer */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1296 int nallocated; /* number of result bytes allocated */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1297 int nneeded; /* number of result bytes needed */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1298 char stackbuf[MAX_SHORT_UNICHARS * 4];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1299
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1300 assert(s != NULL);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1301 assert(size >= 0);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1302
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1303 if (size <= MAX_SHORT_UNICHARS) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1304 /* Write into the stack buffer; nallocated can't overflow.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1305 * At the end, we'll allocate exactly as much heap space as it
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1306 * turns out we need.
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1307 */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1308 nallocated = Py_SAFE_DOWNCAST(sizeof(stackbuf), size_t, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1309 v = NULL; /* will allocate after we're done */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1310 p = stackbuf;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1311 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1312 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1313 /* Overallocate on the heap, and give the excess back at the end. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1314 nallocated = size * 4;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1315 if (nallocated / 4 != size) /* overflow! */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1316 return PyErr_NoMemory();
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1317 v = PyString_FromStringAndSize(NULL, nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1318 if (v == NULL)
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1319 return NULL;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1320 p = PyString_AS_STRING(v);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1321 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1322
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1323 for (i = 0; i < size;) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1324 Py_UCS4 ch = s[i++];
21882 94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding Marc-André Lemburg <mal@egenix.com> parents: 21262 diff changeset	1325
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1326 if (ch < 0x80)
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1327 /* Encode ASCII */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1328 *p++ = (char) ch;
21882 94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding Marc-André Lemburg <mal@egenix.com> parents: 21262 diff changeset	1329
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1330 else if (ch < 0x0800) {
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1331 /* Encode Latin-1 */
21884 04cf4e13e5c9 Cosmetics. Marc-André Lemburg <mal@egenix.com> parents: 21883 diff changeset	1332 *p++ = (char)(0xc0 \| (ch >> 6));
04cf4e13e5c9 Cosmetics. Marc-André Lemburg <mal@egenix.com> parents: 21883 diff changeset	1333 *p++ = (char)(0x80 \| (ch & 0x3f));
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1334 }
21882 94bac88f150f Fix for the UTF-8 memory allocation bug and the UTF-8 encoding Marc-André Lemburg <mal@egenix.com> parents: 21262 diff changeset	1335 else {
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1336 /* Encode UCS2 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1337 if (ch < 0x10000) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1338 /* Special case: check for high surrogate */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1339 if (0xD800 <= ch && ch <= 0xDBFF && i != size) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1340 Py_UCS4 ch2 = s[i];
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1341 /* Check for low surrogate and combine the two to
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1342 form a UCS4 value */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1343 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1344 ch = ((ch - 0xD800) << 10 \| (ch2 - 0xDC00)) + 0x10000;
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1345 i++;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1346 goto encodeUCS4;
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1347 }
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1348 /* Fall through: handles isolated high surrogates */
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1349 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1350 *p++ = (char)(0xe0 \| (ch >> 12));
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1351 *p++ = (char)(0x80 \| ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1352 *p++ = (char)(0x80 \| (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1353 continue;
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1354 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1355 encodeUCS4:
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1356 /* Encode UCS4 Unicode ordinals */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1357 *p++ = (char)(0xf0 \| (ch >> 18));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1358 *p++ = (char)(0x80 \| ((ch >> 12) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1359 *p++ = (char)(0x80 \| ((ch >> 6) & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1360 *p++ = (char)(0x80 \| (ch & 0x3f));
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1361 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1362 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1363
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1364 if (v == NULL) {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1365 /* This was stack allocated. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1366 nneeded = Py_SAFE_DOWNCAST(p - stackbuf, long, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1367 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1368 v = PyString_FromStringAndSize(stackbuf, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1369 }
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1370 else {
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1371 /* Cut back to size actually needed. */
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1372 nneeded = Py_SAFE_DOWNCAST(p - PyString_AS_STRING(v), long, int);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1373 assert(nneeded <= nallocated);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1374 _PyString_Resize(&v, nneeded);
b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1375 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1376 return v;
23203 583c226afde6 Back out 2.140. Martin v. Löwis <martin@v.loewis.de> parents: 23198 diff changeset	1377
23294 b735e89ccda7 SF patch 549375: Compromise PyUnicode_EncodeUTF8 Tim Peters <tim.peters@gmail.com> parents: 23228 diff changeset	1378 #undef MAX_SHORT_UNICHARS
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1379 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1380
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1381 PyObject PyUnicode_AsUTF8String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1382 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1383 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1384 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1385 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1386 }
14059 d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1387 return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1388 PyUnicode_GET_SIZE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1389 NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1390 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1391
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1392 /* --- UTF-16 Codec ------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1393
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1394 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1395 PyUnicode_DecodeUTF16(const char *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1396 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1397 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1398 int *byteorder)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1399 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1400 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1401 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1402 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1403 int outpos;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1404 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1405 Py_UNICODE *p;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1406 const unsigned char q, e;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1407 int bo = 0; /* assume native ordering by default */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1408 const char *errmsg = "";
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1409 /* Offsets from q for retrieving byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1410 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1411 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1412 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1413 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1414 #endif
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1415 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1416 PyObject *exc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1417
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1418 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1419 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1420 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1421 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1422 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1423 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1424 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1425
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1426 /* Unpack UTF-16 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1427 p = unicode->str;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1428 q = (unsigned char *)s;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1429 e = q + size;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1430
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1431 if (byteorder)
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1432 bo = *byteorder;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1433
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1434 /* Check for BOM marks (U+FEFF) in the input and adjust current
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1435 byte order setting accordingly. In native mode, the leading BOM
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1436 mark is skipped, in all other modes, it is copied to the output
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1437 stream as-is (giving a ZWNBSP character). */
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1438 if (bo == 0) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1439 const Py_UNICODE bom = (q[ihi] << 8) \| q[ilo];
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1440 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1441 if (bom == 0xFEFF) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1442 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1443 bo = -1;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1444 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1445 else if (bom == 0xFFFE) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1446 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1447 bo = 1;
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1448 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1449 #else
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1450 if (bom == 0xFEFF) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1451 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1452 bo = 1;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1453 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1454 else if (bom == 0xFFFE) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1455 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1456 bo = -1;
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1457 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1458 #endif
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1459 }
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1460
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1461 if (bo == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1462 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1463 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1464 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1465 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1466 else if (bo == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1467 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1468 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1469 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1470 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1471
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1472 while (q < e) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1473 Py_UNICODE ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1474 /* remaing bytes at the end? (size should be even) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1475 if (e-q<2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1476 errmsg = "truncated data";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1477 startinpos = ((const char *)q)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1478 endinpos = ((const char *)e)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1479 goto utf16Error;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1480 /* The remaining input chars are ignored if the callback
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1481 chooses to skip the input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1482 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1483 ch = (q[ihi] << 8) \| q[ilo];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1484
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1485 q += 2;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1486
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1487 if (ch < 0xD800 \|\| ch > 0xDFFF) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1488 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1489 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1490 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1491
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1492 /* UTF-16 code pair: */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1493 if (q >= e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1494 errmsg = "unexpected end of data";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1495 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1496 endinpos = ((const char *)e)-starts;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1497 goto utf16Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1498 }
18295 20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness Martin v. Löwis <martin@v.loewis.de> parents: 18292 diff changeset	1499 if (0xD800 <= ch && ch <= 0xDBFF) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1500 Py_UNICODE ch2 = (q[ihi] << 8) \| q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1501 q += 2;
18295 20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness Martin v. Löwis <martin@v.loewis.de> parents: 18292 diff changeset	1502 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1503 #ifndef Py_UNICODE_WIDE
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1504 *p++ = ch;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1505 *p++ = ch2;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1506 #else
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1507 *p++ = (((ch & 0x3FF)<<10) \| (ch2 & 0x3FF)) + 0x10000;
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1508 #endif
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1509 continue;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1510 }
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1511 else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1512 errmsg = "illegal UTF-16 surrogate";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1513 startinpos = (((const char *)q)-4)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1514 endinpos = startinpos+2;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1515 goto utf16Error;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1516 }
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1517
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1518 }
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1519 errmsg = "illegal encoding";
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1520 startinpos = (((const char *)q)-2)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1521 endinpos = startinpos+2;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1522 /* Fall through to report the error */
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1523
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1524 utf16Error:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1525 outpos = p-PyUnicode_AS_UNICODE(unicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1526 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1527 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1528 "utf16", errmsg,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1529 starts, size, &startinpos, &endinpos, &exc, (const char **)&q,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1530 (PyObject **)&unicode, &outpos, &p))
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1531 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1532 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1533
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1534 if (byteorder)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1535 *byteorder = bo;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1536
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1537 /* Adjust length */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	1538 if (_PyUnicode_Resize(&unicode, p - unicode->str))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1539 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1540
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1541 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1542 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1543 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1544
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1545 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1546 Py_DECREF(unicode);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1547 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1548 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1549 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1550 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1551
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1552 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1553 PyUnicode_EncodeUTF16(const Py_UNICODE *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1554 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1555 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1556 int byteorder)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1557 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1558 PyObject *v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1559 unsigned char *p;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1560 int i, pairs;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1561 /* Offsets from p for storing byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1562 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1563 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1564 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1565 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1566 #endif
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1567
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1568 #define STORECHAR(CH) \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1569 do { \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1570 p[ihi] = ((CH) >> 8) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1571 p[ilo] = (CH) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1572 p += 2; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1573 } while(0)
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1574
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1575 for (i = pairs = 0; i < size; i++)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1576 if (s[i] >= 0x10000)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1577 pairs++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1578 v = PyString_FromStringAndSize(NULL,
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1579 2 * (size + pairs + (byteorder == 0)));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1580 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1581 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1582
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1583 p = (unsigned char *)PyString_AS_STRING(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1584 if (byteorder == 0)
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1585 STORECHAR(0xFEFF);
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1586 if (size == 0)
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	1587 return v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1588
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1589 if (byteorder == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1590 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1591 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1592 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1593 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1594 else if (byteorder == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1595 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1596 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1597 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1598 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1599
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1600 while (size-- > 0) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1601 Py_UNICODE ch = *s++;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1602 Py_UNICODE ch2 = 0;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1603 if (ch >= 0x10000) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1604 ch2 = 0xDC00 \| ((ch-0x10000) & 0x3FF);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1605 ch = 0xD800 \| ((ch-0x10000) >> 10);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1606 }
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1607 STORECHAR(ch);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1608 if (ch2)
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1609 STORECHAR(ch2);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1610 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1611 return v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1612 #undef STORECHAR
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1613 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1614
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1615 PyObject PyUnicode_AsUTF16String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1616 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1617 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1618 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1619 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1620 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1621 return PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1622 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1623 NULL,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1624 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1625 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1626
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1627 /* --- Unicode Escape Codec ----------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1628
16200 3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1629 static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL;
13063 eb243443b884 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13049 diff changeset	1630
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1631 PyObject PyUnicode_DecodeUnicodeEscape(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1632 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1633 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1634 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1635 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1636 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1637 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1638 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1639 int i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1640 PyUnicodeObject *v;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1641 Py_UNICODE *p;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1642 const char *end;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1643 char* message;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1644 Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1645 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1646 PyObject *exc = NULL;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1647
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1648 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1649 Unicode string, so we start with size here and then reduce the
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1650 length after conversion to the true value.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1651 (but if the error callback returns a long replacement string
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1652 we'll have to allocate more space) */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1653 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1654 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1655 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1656 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1657 return (PyObject *)v;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1658
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1659 p = PyUnicode_AS_UNICODE(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1660 end = s + size;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1661
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1662 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1663 unsigned char c;
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1664 Py_UNICODE x;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1665 int digits;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1666
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1667 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1668 if (*s != '\\') {
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1669 p++ = (unsigned char) s++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1670 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1671 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1672
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1673 startinpos = s-starts;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1674 /* \ - Escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1675 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1676 switch (*s++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1677
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1678 /* \x escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1679 case '\n': break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1680 case '\\': *p++ = '\\'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1681 case '\'': *p++ = '\''; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1682 case '\"': *p++ = '\"'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1683 case 'b': *p++ = '\b'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1684 case 'f': p++ = '\014'; break; / FF */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1685 case 't': *p++ = '\t'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1686 case 'n': *p++ = '\n'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1687 case 'r': *p++ = '\r'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1688 case 'v': p++ = '\013'; break; / VT */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1689 case 'a': p++ = '\007'; break; / BEL, not classic C */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1690
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1691 /* \OOO (octal) escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1692 case '0': case '1': case '2': case '3':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1693 case '4': case '5': case '6': case '7':
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1694 x = s[-1] - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1695 if ('0' <= s && s <= '7') {
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1696 x = (x<<3) + *s++ - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1697 if ('0' <= s && s <= '7')
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1698 x = (x<<3) + *s++ - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1699 }
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1700 *p++ = x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1701 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1702
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1703 /* hex escapes */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1704 /* \xXX */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1705 case 'x':
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1706 digits = 2;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1707 message = "truncated \\xXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1708 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1709
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1710 /* \uXXXX */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1711 case 'u':
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1712 digits = 4;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1713 message = "truncated \\uXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1714 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1715
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1716 /* \UXXXXXXXX */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1717 case 'U':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1718 digits = 8;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1719 message = "truncated \\UXXXXXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1720 hexescape:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1721 chr = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1722 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1723 if (s+digits>end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1724 endinpos = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1725 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1726 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1727 "unicodeescape", "end of string in escape sequence",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1728 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1729 (PyObject **)&v, &outpos, &p))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1730 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1731 goto nextByte;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1732 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1733 for (i = 0; i < digits; ++i) {
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1734 c = (unsigned char) s[i];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1735 if (!isxdigit(c)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1736 endinpos = (s+i+1)-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1737 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1738 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1739 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1740 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1741 (PyObject **)&v, &outpos, &p))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1742 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1743 goto nextByte;
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1744 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1745 chr = (chr<<4) & ~0xF;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1746 if (c >= '0' && c <= '9')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1747 chr += c - '0';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1748 else if (c >= 'a' && c <= 'f')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1749 chr += 10 + c - 'a';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1750 else
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1751 chr += 10 + c - 'A';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1752 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1753 s += i;
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1754 if (chr == 0xffffffff)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1755 /* _decoding_error will have already written into the
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1756 target buffer. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1757 break;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1758 store:
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1759 /* when we get here, chr is a 32-bit unicode character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1760 if (chr <= 0xffff)
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1761 /* UCS-2 character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1762 *p++ = (Py_UNICODE) chr;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1763 else if (chr <= 0x10ffff) {
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1764 /* UCS-4 character. Either store directly, or as
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1765 surrogate pair. */
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1766 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1767 *p++ = chr;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1768 #else
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1769 chr -= 0x10000L;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1770 *p++ = 0xD800 + (Py_UNICODE) (chr >> 10);
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	1771 *p++ = 0xDC00 + (Py_UNICODE) (chr & 0x03FF);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1772 #endif
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1773 } else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1774 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1775 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1776 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1777 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1778 "unicodeescape", "illegal Unicode character",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1779 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1780 (PyObject **)&v, &outpos, &p))
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1781 goto onError;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1782 }
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1783 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1784
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1785 /* \N{name} */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1786 case 'N':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1787 message = "malformed \\N character escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1788 if (ucnhash_CAPI == NULL) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1789 /* load the unicode data module */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1790 PyObject m, v;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1791 m = PyImport_ImportModule("unicodedata");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1792 if (m == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1793 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1794 v = PyObject_GetAttrString(m, "ucnhash_CAPI");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1795 Py_DECREF(m);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1796 if (v == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1797 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1798 ucnhash_CAPI = PyCObject_AsVoidPtr(v);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1799 Py_DECREF(v);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1800 if (ucnhash_CAPI == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1801 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1802 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1803 if (*s == '{') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1804 const char *start = s+1;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1805 /* look for the closing brace */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1806 while (*s != '}' && s < end)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1807 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1808 if (s > start && s < end && *s == '}') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1809 /* found a name. look it up in the unicode database */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1810 message = "unknown Unicode character name";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1811 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1812 if (ucnhash_CAPI->getcode(start, s-start-1, &chr))
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1813 goto store;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1814 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1815 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1816 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1817 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1818 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1819 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1820 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1821 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1822 (PyObject **)&v, &outpos, &p))
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1823 goto onError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1824 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1825
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1826 default:
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1827 if (s > end) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1828 message = "\\ at end of string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1829 s--;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1830 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1831 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1832 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1833 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1834 "unicodeescape", message,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1835 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1836 (PyObject **)&v, &outpos, &p))
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1837 goto onError;
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1838 }
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1839 else {
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1840 *p++ = '\\';
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1841 *p++ = (unsigned char)s[-1];
795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1842 }
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1843 break;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1844 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1845 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1846 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1847 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1848 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1849 goto onError;
30161 54617ed2d830 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 30150 diff changeset	1850 Py_XDECREF(errorHandler);
54617ed2d830 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 30150 diff changeset	1851 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1852 return (PyObject *)v;
22521 795bdd692176 Fix whitespace. Walter Dörwald <walter@livinglogic.de> parents: 22446 diff changeset	1853
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1854 ucnhashError:
16200 3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1855 PyErr_SetString(
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1856 PyExc_UnicodeError,
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1857 "\\N escapes not supported (can't load unicodedata module)"
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1858 );
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1859 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1860 Py_XDECREF(exc);
16064 a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute Fredrik Lundh <fredrik@pythonware.com> parents: 16029 diff changeset	1861 return NULL;
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute Fredrik Lundh <fredrik@pythonware.com> parents: 16029 diff changeset	1862
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1863 onError:
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1864 Py_XDECREF(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1865 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	1866 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1867 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1868 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1869
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1870 /* Return a Unicode-Escape string version of the Unicode object.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1871
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1872 If quotes is true, the string is enclosed in u"" or u'' quotes as
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1873 appropriate.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1874
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1875 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1876
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1877 static const Py_UNICODE findchar(const Py_UNICODE s,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1878 int size,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1879 Py_UNICODE ch);
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1880
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1881 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1882 PyObject unicodeescape_string(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1883 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1884 int quotes)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1885 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1886 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1887 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1888
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1889 static const char *hexdigit = "0123456789abcdef";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1890
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1891 repr = PyString_FromStringAndSize(NULL, 2 + 6*size + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1892 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1893 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1894
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1895 p = PyString_AS_STRING(repr);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1896
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1897 if (quotes) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1898 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1899 *p++ = (findchar(s, size, '\'') &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1900 !findchar(s, size, '"')) ? '"' : '\'';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1901 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1902 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1903 Py_UNICODE ch = *s++;
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1904
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1905 /* Escape quotes */
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1906 if (quotes &&
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1907 (ch == (Py_UNICODE) PyString_AS_STRING(repr)[1] \|\| ch == '\\')) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1908 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1909 *p++ = (char) ch;
20018 8aa1ada33f64 Fix a bug in rendering of \\ by repr() -- it rendered as \\\ instead Guido van Rossum <guido@python.org> parents: 20007 diff changeset	1910 continue;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1911 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1912
18773 743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This Guido van Rossum <guido@python.org> parents: 18324 diff changeset	1913 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1914 /* Map 21-bit characters to '\U00xxxxxx' */
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1915 else if (ch >= 0x10000) {
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1916 int offset = p - PyString_AS_STRING(repr);
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1917
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1918 /* Resize the string if necessary */
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1919 if (offset + 12 > PyString_GET_SIZE(repr)) {
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1920 if (_PyString_Resize(&repr, PyString_GET_SIZE(repr) + 100))
23295 6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people Tim Peters <tim.peters@gmail.com> parents: 23294 diff changeset	1921 return NULL;
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1922 p = PyString_AS_STRING(repr) + offset;
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1923 }
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1924
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1925 *p++ = '\\';
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1926 *p++ = 'U';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1927 *p++ = hexdigit[(ch >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1928 *p++ = hexdigit[(ch >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1929 *p++ = hexdigit[(ch >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1930 *p++ = hexdigit[(ch >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1931 *p++ = hexdigit[(ch >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1932 *p++ = hexdigit[(ch >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1933 *p++ = hexdigit[(ch >> 4) & 0x0000000F];
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1934 *p++ = hexdigit[ch & 0x0000000F];
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1935 continue;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1936 }
18773 743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This Guido van Rossum <guido@python.org> parents: 18324 diff changeset	1937 #endif
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1938 /* Map UTF-16 surrogate pairs to Unicode \UXXXXXXXX escapes */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1939 else if (ch >= 0xD800 && ch < 0xDC00) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1940 Py_UNICODE ch2;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1941 Py_UCS4 ucs;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1942
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1943 ch2 = *s++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1944 size--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1945 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1946 ucs = (((ch & 0x03FF) << 10) \| (ch2 & 0x03FF)) + 0x00010000;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1947 *p++ = '\\';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1948 *p++ = 'U';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1949 *p++ = hexdigit[(ucs >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1950 *p++ = hexdigit[(ucs >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1951 *p++ = hexdigit[(ucs >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1952 *p++ = hexdigit[(ucs >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1953 *p++ = hexdigit[(ucs >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1954 *p++ = hexdigit[(ucs >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1955 *p++ = hexdigit[(ucs >> 4) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1956 *p++ = hexdigit[ucs & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1957 continue;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1958 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1959 /* Fall through: isolated surrogates are copied as-is */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1960 s--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1961 size++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1962 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1963
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1964 /* Map 16-bit characters to '\uxxxx' */
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1965 if (ch >= 256) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1966 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1967 *p++ = 'u';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1968 *p++ = hexdigit[(ch >> 12) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1969 *p++ = hexdigit[(ch >> 8) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1970 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1971 *p++ = hexdigit[ch & 0x000F];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1972 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1973
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1974 /* Map special whitespace to '\t', \n', '\r' */
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1975 else if (ch == '\t') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1976 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1977 *p++ = 't';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1978 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1979 else if (ch == '\n') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1980 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1981 *p++ = 'n';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1982 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1983 else if (ch == '\r') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1984 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1985 *p++ = 'r';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1986 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1987
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1988 /* Map non-printable US ASCII to '\xhh' */
21067 c9c4c523dcbf Fix for bug #485951: repr diff between string and unicode. Marc-André Lemburg <mal@egenix.com> parents: 21028 diff changeset	1989 else if (ch < ' ' \|\| ch >= 0x7F) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1990 *p++ = '\\';
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1991 *p++ = 'x';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1992 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1993 *p++ = hexdigit[ch & 0x000F];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1994 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1995
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1996 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1997 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1998 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1999 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2000 if (quotes)
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	2001 *p++ = PyString_AS_STRING(repr)[1];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2002
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2003 *p = '\0';
23295 6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people Tim Peters <tim.peters@gmail.com> parents: 23294 diff changeset	2004 _PyString_Resize(&repr, p - PyString_AS_STRING(repr));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2005 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2006 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2007
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2008 PyObject PyUnicode_EncodeUnicodeEscape(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2009 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2010 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2011 return unicodeescape_string(s, size, 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2012 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2013
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2014 PyObject PyUnicode_AsUnicodeEscapeString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2015 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2016 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2017 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2018 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2019 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2020 return PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2021 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2022 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2023
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2024 /* --- Raw Unicode Escape Codec ------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2025
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2026 PyObject PyUnicode_DecodeRawUnicodeEscape(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2027 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2028 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2029 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2030 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2031 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2032 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2033 int outpos;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2034 PyUnicodeObject *v;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2035 Py_UNICODE *p;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2036 const char *end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2037 const char *bs;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2038 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2039 PyObject *exc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2040
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2041 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2042 Unicode string, so we start with size here and then reduce the
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2043 length after conversion to the true value. (But decoding error
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2044 handler might have to resize the string) */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2045 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2046 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2047 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2048 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2049 return (PyObject *)v;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2050 p = PyUnicode_AS_UNICODE(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2051 end = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2052 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2053 unsigned char c;
22423 b523a470a0ce Do not insert characters for unicode-escape decoders if the error mode Martin v. Löwis <martin@v.loewis.de> parents: 22087 diff changeset	2054 Py_UCS4 x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2055 int i;
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2056 int count;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2057
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2058 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2059 if (*s != '\\') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2060 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2061 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2062 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2063 startinpos = s-starts;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2064
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2065 /* \u-escapes are only interpreted iff the number of leading
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2066 backslashes if odd */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2067 bs = s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2068 for (;s < end;) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2069 if (*s != '\\')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2070 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2071 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2072 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2073 if (((s - bs) & 1) == 0 \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2074 s >= end \|\|
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2075 (s != 'u' && s != 'U')) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2076 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2077 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2078 p--;
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2079 count = *s=='u' ? 4 : 8;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2080 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2081
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2082 /* \uXXXX with 4 hex digits, \Uxxxxxxxx with 8 */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2083 outpos = p-PyUnicode_AS_UNICODE(v);
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2084 for (x = 0, i = 0; i < count; ++i, ++s) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2085 c = (unsigned char)*s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2086 if (!isxdigit(c)) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2087 endinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2088 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2089 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2090 "rawunicodeescape", "truncated \\uXXXX",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2091 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2092 (PyObject **)&v, &outpos, &p))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2093 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2094 goto nextByte;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2095 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2096 x = (x<<4) & ~0xF;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2097 if (c >= '0' && c <= '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2098 x += c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2099 else if (c >= 'a' && c <= 'f')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2100 x += 10 + c - 'a';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2101 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2102 x += 10 + c - 'A';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2103 }
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2104 #ifndef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2105 if (x > 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2106 if (unicode_decode_call_errorhandler(
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2107 errors, &errorHandler,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2108 "rawunicodeescape", "\\Uxxxxxxxx out of range",
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2109 starts, size, &startinpos, &endinpos, &exc, &s,
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2110 (PyObject **)&v, &outpos, &p))
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2111 goto onError;
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2112 }
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2113 #endif
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2114 *p++ = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2115 nextByte:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2116 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2117 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2118 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2119 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2120 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2121 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2122 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2123
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2124 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2125 Py_XDECREF(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2126 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2127 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2128 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2129 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2130
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2131 PyObject PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2132 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2133 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2134 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2135 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2136 char *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2137
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	2138 static const char *hexdigit = "0123456789abcdef";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2139
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2140 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2141 repr = PyString_FromStringAndSize(NULL, 10 * size);
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2142 #else
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2143 repr = PyString_FromStringAndSize(NULL, 6 * size);
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2144 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2145 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2146 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2147 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2148 return repr;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2149
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2150 p = q = PyString_AS_STRING(repr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2151 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2152 Py_UNICODE ch = *s++;
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2153 #ifdef Py_UNICODE_WIDE
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2154 /* Map 32-bit characters to '\Uxxxxxxxx' */
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2155 if (ch >= 0x10000) {
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2156 *p++ = '\\';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2157 *p++ = 'U';
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2158 *p++ = hexdigit[(ch >> 28) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2159 *p++ = hexdigit[(ch >> 24) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2160 *p++ = hexdigit[(ch >> 20) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2161 *p++ = hexdigit[(ch >> 16) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2162 *p++ = hexdigit[(ch >> 12) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2163 *p++ = hexdigit[(ch >> 8) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2164 *p++ = hexdigit[(ch >> 4) & 0xf];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2165 *p++ = hexdigit[ch & 15];
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2166 }
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2167 else
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	2168 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2169 /* Map 16-bit characters to '\uxxxx' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2170 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2171 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2172 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2173 *p++ = hexdigit[(ch >> 12) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2174 *p++ = hexdigit[(ch >> 8) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2175 *p++ = hexdigit[(ch >> 4) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2176 *p++ = hexdigit[ch & 15];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2177 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2178 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2179 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2180 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2181 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2182 *p = '\0';
23295 6fa76b4e59fe Repair widespread misuse of _PyString_Resize. Since it's clear people Tim Peters <tim.peters@gmail.com> parents: 23294 diff changeset	2183 _PyString_Resize(&repr, p - q);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2184 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2185 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2186
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2187 PyObject PyUnicode_AsRawUnicodeEscapeString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2188 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2189 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2190 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2191 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2192 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2193 return PyUnicode_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2194 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2195 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2196
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2197 /* --- Latin-1 Codec ------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2198
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2199 PyObject PyUnicode_DecodeLatin1(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2200 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2201 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2202 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2203 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2204 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2205
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2206 /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2207 if (size == 1 && (unsigned char)s < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2208 Py_UNICODE r = (unsigned char)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2209 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2210 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2211
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2212 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2213 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2214 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2215 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2216 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2217 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2218 while (size-- > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2219 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2220 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2221
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2222 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2223 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2224 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2225 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2226
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2227 /* create or adjust a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2228 static void make_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2229 const char *encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2230 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2231 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2232 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2233 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2234 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2235 *exceptionObject = PyUnicodeEncodeError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2236 encoding, unicode, size, startpos, endpos, reason);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2237 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2238 else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2239 if (PyUnicodeEncodeError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2240 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2241 if (PyUnicodeEncodeError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2242 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2243 if (PyUnicodeEncodeError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2244 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2245 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2246 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2247 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2248 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2249 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2250 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2251
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2252 /* raises a UnicodeEncodeError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2253 static void raise_encode_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2254 const char *encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2255 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2256 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2257 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2258 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2259 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2260 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2261 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2262 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2263 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2264
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2265 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2266 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2267 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2268 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2269 static PyObject unicode_encode_call_errorhandler(const char errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2270 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2271 const char encoding, const char reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2272 const Py_UNICODE unicode, int size, PyObject *exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2273 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2274 int *newpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2275 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2276 static char *argparse = "O!i;encoding error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2277
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2278 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2279 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2280
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2281 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2282 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2283 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2284 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2285 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2286
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2287 make_encode_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2288 encoding, unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2289 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2290 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2291
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2292 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2293 errorHandler, exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2294 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2295 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2296 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2297 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2298 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2299 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2300 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2301 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2302 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2303 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2304 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2305 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2306 if (*newpos<0)
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2307 newpos = size+newpos;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2308 if (newpos<0 \|\| newpos>size) {
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2309 PyErr_Format(PyExc_IndexError, "position %d from error handler out of bounds", *newpos);
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2310 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2311 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	2312 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2313 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2314 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2315 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2316 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2317
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2318 static PyObject unicode_encode_ucs1(const Py_UNICODE p,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2319 int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2320 const char *errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2321 int limit)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2322 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2323 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2324 PyObject *res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2325 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2326 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2327 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2328 /* pointer to the beginning of the unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2329 /* const Py_UNICODE badp = NULL; /
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2330 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2331 char *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2332 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2333 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2334 int ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2335 char *encoding = (limit == 256) ? "latin-1" : "ascii";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2336 char *reason = (limit == 256) ? "ordinal not in range(256)" : "ordinal not in range(128)";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2337 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2338 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2339 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2340 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2341 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2342
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2343 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2344 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2345 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2346 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2347 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2348 if (size == 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2349 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2350 str = PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2351 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2352
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2353 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2354 Py_UNICODE c = *p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2355
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2356 /* can we encode this? */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2357 if (c<limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2358 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2359 *str++ = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2360 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2361 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2362 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2363 int unicodepos = p-startp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2364 int requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2365 PyObject *repunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2366 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2367 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2368 int respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2369 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2370 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2371 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2372 const Py_UNICODE *collend = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2373 /* find all unecodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2374 while ((collend < endp) && ((*collend)>=limit))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2375 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2376 /* cache callback name lookup (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2377 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2378 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2379 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2380 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2381 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2382 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2383 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2384 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2385 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2386 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2387 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2388 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2389 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2390 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2391 raise_encode_exception(&exc, encoding, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2392 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2393 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2394 while (collstart++<collend)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2395 str++ = '?'; / fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2396 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2397 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2398 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2399 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2400 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2401 /* determine replacement size (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2402 for (p = collstart, repsize = 0; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2403 if (*p<10)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2404 repsize += 2+1+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2405 else if (*p<100)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2406 repsize += 2+2+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2407 else if (*p<1000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2408 repsize += 2+3+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2409 else if (*p<10000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2410 repsize += 2+4+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2411 else if (*p<100000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2412 repsize += 2+5+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2413 else if (*p<1000000)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2414 repsize += 2+6+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2415 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2416 repsize += 2+7+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2417 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2418 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2419 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2420 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2421 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2422 if (_PyString_Resize(&res, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2423 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2424 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2425 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2426 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2427 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2428 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2429 str += sprintf(str, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2430 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2431 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2432 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2433 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2434 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2435 encoding, reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2436 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2437 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2438 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2439 /* need more space? (at least enough for what we
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2440 have+the replacement+the rest of the string, so
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2441 we won't have to check space for encodable characters) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2442 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2443 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2444 requiredsize = respos+repsize+(endp-collend);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2445 if (requiredsize > ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2446 if (requiredsize<2*ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2447 requiredsize = 2*ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2448 if (_PyString_Resize(&res, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2449 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2450 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2451 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2452 str = PyString_AS_STRING(res) + respos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2453 ressize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2454 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2455 /* check if there is anything unencodable in the replacement
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2456 and copy it to the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2457 for (uni2 = PyUnicode_AS_UNICODE(repunicode);repsize-->0; ++uni2, ++str) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2458 c = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2459 if (c >= limit) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2460 raise_encode_exception(&exc, encoding, startp, size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2461 unicodepos, unicodepos+1, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2462 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2463 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2464 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2465 *str = (char)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2466 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2467 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2468 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2469 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2470 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2471 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2472 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2473 respos = str-PyString_AS_STRING(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2474 if (respos<ressize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2475 /* If this falls res will be NULL */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2476 _PyString_Resize(&res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2477 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2478 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2479 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2480
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2481 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2482 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2483 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2484 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2485 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2486 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2487
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2488 PyObject PyUnicode_EncodeLatin1(const Py_UNICODE p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2489 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2490 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2491 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2492 return unicode_encode_ucs1(p, size, errors, 256);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2493 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2494
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2495 PyObject PyUnicode_AsLatin1String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2496 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2497 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2498 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2499 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2500 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2501 return PyUnicode_EncodeLatin1(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2502 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2503 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2504 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2505
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2506 /* --- 7-bit ASCII Codec -------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2507
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2508 PyObject PyUnicode_DecodeASCII(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2509 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2510 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2511 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2512 const char *starts = s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2513 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2514 Py_UNICODE *p;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2515 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2516 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2517 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2518 const char *e;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2519 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2520 PyObject *exc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2521
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2522 /* ASCII is equivalent to the first 128 ordinals in Unicode. */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2523 if (size == 1 && (unsigned char)s < 128) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2524 Py_UNICODE r = (unsigned char)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2525 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2526 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2527
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2528 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2529 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2530 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2531 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2532 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2533 p = PyUnicode_AS_UNICODE(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2534 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2535 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2536 register unsigned char c = (unsigned char)*s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2537 if (c < 128) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2538 *p++ = c;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2539 ++s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2540 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2541 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2542 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2543 endinpos = startinpos + 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2544 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2545 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2546 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2547 "ascii", "ordinal not in range(128)",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2548 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2549 (PyObject **)&v, &outpos, &p))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2550 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2551 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2552 }
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2553 if (p - PyUnicode_AS_UNICODE(v) < PyString_GET_SIZE(v))
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2554 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2555 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2556 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2557 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2558 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2559
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2560 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2561 Py_XDECREF(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2562 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2563 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2564 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2565 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2566
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2567 PyObject PyUnicode_EncodeASCII(const Py_UNICODE p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2568 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2569 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2570 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2571 return unicode_encode_ucs1(p, size, errors, 128);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2572 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2573
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2574 PyObject PyUnicode_AsASCIIString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2575 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2576 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2577 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2578 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2579 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2580 return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2581 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2582 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2583 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2584
24071 8142f8d668a0 Patch #569753: Remove support for WIN16. Martin v. Löwis <martin@v.loewis.de> parents: 23837 diff changeset	2585 #if defined(MS_WINDOWS) && defined(HAVE_USABLE_WCHAR_T)
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2586
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2587 /* --- MBCS codecs for Windows -------------------------------------------- */
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2588
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2589 PyObject PyUnicode_DecodeMBCS(const char s,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2590 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2591 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2592 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2593 PyUnicodeObject *v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2594 Py_UNICODE *p;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2595
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2596 /* First get the size of the result */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2597 DWORD usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0);
12582 0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2598 if (size > 0 && usize==0)
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2599 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2600
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2601 v = _PyUnicode_New(usize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2602 if (v == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2603 return NULL;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2604 if (usize == 0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2605 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2606 p = PyUnicode_AS_UNICODE(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2607 if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2608 Py_DECREF(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2609 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2610 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2611
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2612 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2613 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2614
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2615 PyObject PyUnicode_EncodeMBCS(const Py_UNICODE p,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2616 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2617 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2618 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2619 PyObject *repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2620 char *s;
12582 0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2621 DWORD mbcssize;
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2622
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2623 /* If there are no characters, bail now! */
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2624 if (size==0)
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2625 return PyString_FromString("");
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2626
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2627 /* First get the size of the result */
12582 0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2628 mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL);
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2629 if (mbcssize==0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2630 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2631
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2632 repr = PyString_FromStringAndSize(NULL, mbcssize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2633 if (repr == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2634 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2635 if (mbcssize == 0)
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2636 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2637
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2638 /* Do the conversion */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2639 s = PyString_AS_STRING(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2640 if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2641 Py_DECREF(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2642 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2643 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2644 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2645 }
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2646
29706 a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2647 PyObject PyUnicode_AsMBCSString(PyObject unicode)
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2648 {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2649 if (!PyUnicode_Check(unicode)) {
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2650 PyErr_BadArgument();
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2651 return NULL;
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2652 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2653 return PyUnicode_EncodeMBCS(PyUnicode_AS_UNICODE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2654 PyUnicode_GET_SIZE(unicode),
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2655 NULL);
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2656 }
a4fead0e24f8 Support 'mbcs' as a 'built-in' encoding, so the C API can use it without Mark Hammond <mhammond@skippinet.com.au> parents: 29641 diff changeset	2657
24071 8142f8d668a0 Patch #569753: Remove support for WIN16. Martin v. Löwis <martin@v.loewis.de> parents: 23837 diff changeset	2658 #endif /* MS_WINDOWS */
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2659
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2660 /* --- Character Mapping Codec -------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2661
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2662 PyObject PyUnicode_DecodeCharmap(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2663 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2664 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2665 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2666 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2667 const char *starts = s;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2668 int startinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2669 int endinpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2670 int outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2671 const char *e;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2672 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2673 Py_UNICODE *p;
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2674 int extrachars = 0;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2675 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2676 PyObject *exc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2677
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2678 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2679 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2680 return PyUnicode_DecodeLatin1(s, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2681
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2682 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2683 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2684 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2685 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2686 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2687 p = PyUnicode_AS_UNICODE(v);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2688 e = s + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2689 while (s < e) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2690 unsigned char ch = *s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2691 PyObject w, x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2692
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2693 /* Get mapping (char ordinal -> integer, Unicode char or None) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2694 w = PyInt_FromLong((long)ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2695 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2696 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2697 x = PyObject_GetItem(mapping, w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2698 Py_DECREF(w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2699 if (x == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2700 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2701 /* No mapping found means: mapping is undefined. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2702 PyErr_Clear();
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2703 x = Py_None;
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2704 Py_INCREF(x);
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2705 } else
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2706 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2707 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2708
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2709 /* Apply mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2710 if (PyInt_Check(x)) {
13390 62754b9ec7df Fixed a couple of places where 'int' was used where 'long' Marc-André Lemburg <mal@egenix.com> parents: 13356 diff changeset	2711 long value = PyInt_AS_LONG(x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2712 if (value < 0 \|\| value > 65535) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2713 PyErr_SetString(PyExc_TypeError,
12909 be36f103a953 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12902 diff changeset	2714 "character mapping must be in range(65536)");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2715 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2716 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2717 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2718 *p++ = (Py_UNICODE)value;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2719 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2720 else if (x == Py_None) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2721 /* undefined mapping */
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2722 outpos = p-PyUnicode_AS_UNICODE(v);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2723 startinpos = s-starts;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2724 endinpos = startinpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2725 if (unicode_decode_call_errorhandler(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2726 errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2727 "charmap", "character maps to <undefined>",
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2728 starts, size, &startinpos, &endinpos, &exc, &s,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2729 (PyObject **)&v, &outpos, &p)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2730 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2731 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2732 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2733 continue;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2734 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2735 else if (PyUnicode_Check(x)) {
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2736 int targetsize = PyUnicode_GET_SIZE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2737
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2738 if (targetsize == 1)
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2739 /* 1-1 mapping */
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2740 p++ = PyUnicode_AS_UNICODE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2741
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2742 else if (targetsize > 1) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2743 /* 1-n mapping */
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2744 if (targetsize > extrachars) {
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2745 /* resize first */
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2746 int oldpos = (int)(p - PyUnicode_AS_UNICODE(v));
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2747 int needed = (targetsize - extrachars) + \
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2748 (targetsize << 2);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2749 extrachars += needed;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2750 if (_PyUnicode_Resize(&v,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2751 PyUnicode_GET_SIZE(v) + needed)) {
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2752 Py_DECREF(x);
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2753 goto onError;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2754 }
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2755 p = PyUnicode_AS_UNICODE(v) + oldpos;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2756 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2757 Py_UNICODE_COPY(p,
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2758 PyUnicode_AS_UNICODE(x),
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2759 targetsize);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2760 p += targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2761 extrachars -= targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2762 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2763 /* 1-0 mapping: skip the character */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2764 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2765 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2766 /* wrong return value */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2767 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2768 "character mapping must return integer, None or unicode");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2769 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2770 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2771 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2772 Py_DECREF(x);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2773 ++s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2774 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2775 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2776 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2777 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2778 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2779 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2780 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2781
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2782 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2783 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2784 Py_XDECREF(exc);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2785 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2786 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2787 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2788
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2789 /* Lookup the character ch in the mapping. If the character
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2790 can't be found, Py_None is returned (or NULL, if another
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2791 error occured). */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2792 static PyObject charmapencode_lookup(Py_UNICODE c, PyObject mapping)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2793 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2794 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2795 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2796
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2797 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2798 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2799 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2800 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2801 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2802 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2803 /* No mapping found means: mapping is undefined. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2804 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2805 x = Py_None;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2806 Py_INCREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2807 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2808 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2809 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2810 }
27175 47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping Walter Dörwald <walter@livinglogic.de> parents: 27170 diff changeset	2811 else if (x == Py_None)
47b134bb51c3 Fix charmapencode_lookup(), so that a None value in the mapping Walter Dörwald <walter@livinglogic.de> parents: 27170 diff changeset	2812 return x;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2813 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2814 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2815 if (value < 0 \|\| value > 255) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2816 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2817 "character mapping must be in range(256)");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2818 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2819 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2820 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2821 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2822 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2823 else if (PyString_Check(x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2824 return x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2825 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2826 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2827 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2828 "character mapping must return integer, None or str");
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2829 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2830 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2831 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2832 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2833
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2834 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2835 various state variables. Reallocate the output string if not enough
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2836 space is available. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2837 was put in the output buffer, or Py_None, if the mapping was undefined
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2838 (in which case no character was written) or NULL, if a
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2839 reallocation error ocurred. The called must decref the result */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2840 static
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2841 PyObject charmapencode_output(Py_UNICODE c, PyObject mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2842 PyObject *outobj, int outpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2843 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2844 PyObject *rep = charmapencode_lookup(c, mapping);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2845
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2846 if (rep==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2847 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2848 else if (rep==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2849 return rep;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2850 else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2851 char outstart = PyString_AS_STRING(outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2852 int outsize = PyString_GET_SIZE(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2853 if (PyInt_Check(rep)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2854 int requiredsize = *outpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2855 if (outsize<requiredsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2856 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2857 if (requiredsize < 2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2858 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2859 if (_PyString_Resize(outobj, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2860 Py_DECREF(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2861 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2862 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2863 outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2864 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2865 outstart[(*outpos)++] = (char)PyInt_AS_LONG(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2866 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2867 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2868 const char *repchars = PyString_AS_STRING(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2869 int repsize = PyString_GET_SIZE(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2870 int requiredsize = *outpos+repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2871 if (outsize<requiredsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2872 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2873 if (requiredsize < 2*outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2874 requiredsize = 2*outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2875 if (_PyString_Resize(outobj, requiredsize)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2876 Py_DECREF(rep);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2877 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2878 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2879 outstart = PyString_AS_STRING(*outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2880 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2881 memcpy(outstart + *outpos, repchars, repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2882 *outpos += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2883 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2884 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2885 return rep;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2886 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2887
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2888 /* handle an error in PyUnicode_EncodeCharmap
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2889 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2890 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2891 int charmap_encoding_error(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2892 const Py_UNICODE p, int size, int inpos, PyObject *mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2893 PyObject **exceptionObject,
30150 5686ca297f55 Backport checkins: Walter Dörwald <walter@livinglogic.de> parents: 29706 diff changeset	2894 int known_errorHandler, PyObject errorHandler, const char errors,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2895 PyObject *res, int respos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2896 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2897 PyObject repunicode = NULL; / initialize to prevent gcc warning */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2898 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2899 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2900 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2901 /* startpos for collecting unencodable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2902 int collstartpos = *inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2903 int collendpos = *inpos+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2904 int collpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2905 char *encoding = "charmap";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2906 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2907
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2908 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2909 /* find all unencodable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2910 while (collendpos < size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2911 x = charmapencode_lookup(p[collendpos], mapping);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2912 if (x==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2913 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2914 else if (x!=Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2915 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2916 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2917 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2918 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2919 ++collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2920 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2921 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2922 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2923 if (*known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2924 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2925 *known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2926 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2927 *known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2928 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2929 *known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2930 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2931 *known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2932 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2933 *known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2934 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2935 switch (*known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2936 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2937 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2938 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2939 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2940 for (collpos = collstartpos; collpos<collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2941 x = charmapencode_output('?', mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2942 if (x==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2943 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2944 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2945 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2946 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2947 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2948 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2949 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2950 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2951 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2952 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2953 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2954 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2955 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2956 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2957 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2958 for (collpos = collstartpos; collpos < collendpos; ++collpos) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2959 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2960 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2961 sprintf(buffer, "&#%d;", (int)p[collpos]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2962 for (cp = buffer; *cp; ++cp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2963 x = charmapencode_output(*cp, mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2964 if (x==NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2965 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2966 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2967 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2968 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2969 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2970 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2971 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2972 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2973 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2974 *inpos = collendpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2975 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2976 default:
30150 5686ca297f55 Backport checkins: Walter Dörwald <walter@livinglogic.de> parents: 29706 diff changeset	2977 repunicode = unicode_encode_call_errorhandler(errors, errorHandler,
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2978 encoding, reason, p, size, exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2979 collstartpos, collendpos, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2980 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2981 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2982 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2983 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2984 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2985 x = charmapencode_output(*uni2, mapping, res, respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2986 if (x==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2987 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2988 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2989 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2990 else if (x==Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2991 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2992 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2993 raise_encode_exception(exceptionObject, encoding, p, size, collstartpos, collendpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2994 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2995 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2996 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2997 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2998 *inpos = newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	2999 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3000 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3001 return 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3002 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3003
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3004 PyObject PyUnicode_EncodeCharmap(const Py_UNICODE p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3005 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3006 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3007 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3008 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3009 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3010 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3011 /* current input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3012 int inpos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3013 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3014 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3015 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3016 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3017 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3018 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3019 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3020 int known_errorHandler = -1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3021
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3022 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3023 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3024 return PyUnicode_EncodeLatin1(p, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3025
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3026 /* allocate enough for a simple encoding without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3027 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3028 res = PyString_FromStringAndSize(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3029 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3030 goto onError;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	3031 if (size == 0)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3032 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3033
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3034 while (inpos<size) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3035 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3036 PyObject *x = charmapencode_output(p[inpos], mapping, &res, &respos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3037 if (x==NULL) /* error */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3038 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3039 if (x==Py_None) { /* unencodable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3040 if (charmap_encoding_error(p, size, &inpos, mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3041 &exc,
30150 5686ca297f55 Backport checkins: Walter Dörwald <walter@livinglogic.de> parents: 29706 diff changeset	3042 &known_errorHandler, &errorHandler, errors,
30163 29b1d0763790 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 30161 diff changeset	3043 &res, &respos)) {
29b1d0763790 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 30161 diff changeset	3044 Py_DECREF(x);
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3045 goto onError;
30163 29b1d0763790 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 30161 diff changeset	3046 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3047 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3048 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3049 /* done with this character => adjust input position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3050 ++inpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3051 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3052 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3053
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3054 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3055 if (respos<PyString_GET_SIZE(res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3056 if (_PyString_Resize(&res, respos))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3057 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3058 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3059 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3060 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3061 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3062
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3063 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3064 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3065 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3066 Py_XDECREF(errorHandler);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3067 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3068 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3069
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3070 PyObject PyUnicode_AsCharmapString(PyObject unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3071 PyObject *mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3072 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3073 if (!PyUnicode_Check(unicode) \|\| mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3074 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3075 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3076 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3077 return PyUnicode_EncodeCharmap(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3078 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3079 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3080 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3081 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3082
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3083 /* create or adjust a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3084 static void make_translate_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3085 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3086 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3087 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3088 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3089 if (*exceptionObject == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3090 *exceptionObject = PyUnicodeTranslateError_Create(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3091 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3092 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3093 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3094 if (PyUnicodeTranslateError_SetStart(*exceptionObject, startpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3095 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3096 if (PyUnicodeTranslateError_SetEnd(*exceptionObject, endpos))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3097 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3098 if (PyUnicodeTranslateError_SetReason(*exceptionObject, reason))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3099 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3100 return;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3101 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3102 Py_DECREF(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3103 *exceptionObject = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3104 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3105 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3106
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3107 /* raises a UnicodeTranslateError */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3108 static void raise_translate_exception(PyObject **exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3109 const Py_UNICODE *unicode, int size,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3110 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3111 const char *reason)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3112 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3113 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3114 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3115 if (*exceptionObject != NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3116 PyCodec_StrictErrors(*exceptionObject);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3117 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3118
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3119 /* error handling callback helper:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3120 build arguments, call the callback and check the arguments,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3121 put the result into newpos and return the replacement string, which
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3122 has to be freed by the caller */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3123 static PyObject unicode_translate_call_errorhandler(const char errors,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3124 PyObject **errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3125 const char *reason,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3126 const Py_UNICODE unicode, int size, PyObject *exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3127 int startpos, int endpos,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3128 int *newpos)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3129 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3130 static char *argparse = "O!i;translating error handler must return (unicode, int) tuple";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3131
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3132 PyObject *restuple;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3133 PyObject *resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3134
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3135 if (*errorHandler == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3136 *errorHandler = PyCodec_LookupError(errors);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3137 if (*errorHandler == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3138 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3139 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3140
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3141 make_translate_exception(exceptionObject,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3142 unicode, size, startpos, endpos, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3143 if (*exceptionObject == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3144 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3145
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3146 restuple = PyObject_CallFunctionObjArgs(
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3147 errorHandler, exceptionObject, NULL);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3148 if (restuple == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3149 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3150 if (!PyTuple_Check(restuple)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3151 PyErr_Format(PyExc_TypeError, &argparse[4]);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3152 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3153 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3154 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3155 if (!PyArg_ParseTuple(restuple, argparse, &PyUnicode_Type,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3156 &resunicode, newpos)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3157 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3158 return NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3159 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3160 if (*newpos<0)
27555 c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3161 newpos = size+newpos;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3162 if (newpos<0 \|\| newpos>size) {
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3163 PyErr_Format(PyExc_IndexError, "position %d from error handler out of bounds", *newpos);
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3164 Py_DECREF(restuple);
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3165 return NULL;
c267f28724dd Change the treatment of positions returned by PEP293 Walter Dörwald <walter@livinglogic.de> parents: 27521 diff changeset	3166 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3167 Py_INCREF(resunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3168 Py_DECREF(restuple);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3169 return resunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3170 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3171
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3172 /* Lookup the character ch in the mapping and put the result in result,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3173 which must be decrefed by the caller.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3174 Return 0 on success, -1 on error */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3175 static
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3176 int charmaptranslate_lookup(Py_UNICODE c, PyObject mapping, PyObject *result)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3177 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3178 PyObject *w = PyInt_FromLong((long)c);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3179 PyObject *x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3180
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3181 if (w == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3182 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3183 x = PyObject_GetItem(mapping, w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3184 Py_DECREF(w);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3185 if (x == NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3186 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3187 /* No mapping found means: use 1:1 mapping. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3188 PyErr_Clear();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3189 *result = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3190 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3191 } else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3192 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3193 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3194 else if (x == Py_None) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3195 *result = x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3196 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3197 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3198 else if (PyInt_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3199 long value = PyInt_AS_LONG(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3200 long max = PyUnicode_GetMax();
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3201 if (value < 0 \|\| value > max) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3202 PyErr_Format(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3203 "character mapping must be in range(0x%lx)", max+1);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3204 Py_DECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3205 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3206 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3207 *result = x;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3208 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3209 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3210 else if (PyUnicode_Check(x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3211 *result = x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3212 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3213 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3214 else {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3215 /* wrong return value */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3216 PyErr_SetString(PyExc_TypeError,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3217 "character mapping must return integer, None or unicode");
30165 4708fa086121 Backport checkin: Walter Dörwald <walter@livinglogic.de> parents: 30163 diff changeset	3218 Py_DECREF(x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3219 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3220 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3221 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3222 /* ensure that *outobj is at least requiredsize characters long,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3223 if not reallocate and adjust various state variables.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3224 Return 0 on success, -1 on error */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3225 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3226 int charmaptranslate_makespace(PyObject outobj, Py_UNICODE outp, int *outsize,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3227 int requiredsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3228 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3229 if (requiredsize > *outsize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3230 /* remember old output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3231 int outpos = outp-PyUnicode_AS_UNICODE(outobj);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3232 /* exponentially overallocate to minimize reallocations */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3233 if (requiredsize < 2 * *outsize)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3234 requiredsize = 2 * *outsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3235 if (_PyUnicode_Resize(outobj, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3236 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3237 outp = PyUnicode_AS_UNICODE(outobj) + outpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3238 *outsize = requiredsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3239 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3240 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3241 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3242 /* lookup the character, put the result in the output string and adjust
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3243 various state variables. Return a new reference to the object that
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3244 was put in the output buffer in *result, or Py_None, if the mapping was
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3245 undefined (in which case no character was written).
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3246 The called must decref result.
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3247 Return 0 on success, -1 on error. */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3248 static
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3249 int charmaptranslate_output(Py_UNICODE c, PyObject *mapping,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3250 PyObject *outobj, int outsize, Py_UNICODE outp, PyObject res)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3251 {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3252 if (charmaptranslate_lookup(c, mapping, res))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3253 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3254 if (*res==NULL) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3255 /* not found => default to 1:1 mapping */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3256 (outp)++ = (Py_UNICODE)c;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3257 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3258 else if (*res==Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3259 ;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3260 else if (PyInt_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3261 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3262 (outp)++ = (Py_UNICODE)PyInt_AS_LONG(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3263 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3264 else if (PyUnicode_Check(*res)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3265 int repsize = PyUnicode_GET_SIZE(*res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3266 if (repsize==1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3267 /* no overflow check, because we know that the space is enough */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3268 (outp)++ = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3269 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3270 else if (repsize!=0) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3271 /* more than one character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3272 int requiredsize = *outsize + repsize - 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3273 if (charmaptranslate_makespace(outobj, outp, outsize, requiredsize))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3274 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3275 memcpy(outp, PyUnicode_AS_UNICODE(res), sizeof(Py_UNICODE)*repsize);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3276 *outp += repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3277 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3278 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3279 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3280 return -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3281 return 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3282 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3283
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3284 PyObject PyUnicode_TranslateCharmap(const Py_UNICODE p,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3285 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3286 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3287 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3288 {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3289 /* output object */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3290 PyObject *res = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3291 /* pointers to the beginning and end+1 of input */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3292 const Py_UNICODE *startp = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3293 const Py_UNICODE *endp = p + size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3294 /* pointer into the output */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3295 Py_UNICODE *str;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3296 /* current output position */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3297 int respos = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3298 int ressize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3299 char *reason = "character maps to <undefined>";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3300 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3301 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3302 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3303 * -1=not initialized, 0=unknown, 1=strict, 2=replace,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3304 * 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3305 int known_errorHandler = -1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3306
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3307 if (mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3308 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3309 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3310 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3311
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3312 /* allocate enough for a simple 1:1 translation without
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3313 replacements, if we need more, we'll resize */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3314 res = PyUnicode_FromUnicode(NULL, size);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3315 if (res == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3316 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3317 if (size == 0)
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3318 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3319 str = PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3320 ressize = size;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3321
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3322 while (p<endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3323 /* try to encode it */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3324 PyObject *x = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3325 if (charmaptranslate_output(*p, mapping, &res, &ressize, &str, &x)) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3326 Py_XDECREF(x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3327 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3328 }
27774 bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	3329 Py_XDECREF(x);
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3330 if (x!=Py_None) /* it worked => adjust input pointer */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3331 ++p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3332 else { /* untranslatable character */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3333 PyObject repunicode = NULL; / initialize to prevent gcc warning */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3334 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3335 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3336 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3337 /* startpos for collecting untranslatable chars */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3338 const Py_UNICODE *collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3339 const Py_UNICODE *collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3340 const Py_UNICODE *coll;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3341
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3342 /* find all untranslatable characters */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3343 while (collend < endp) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3344 if (charmaptranslate_lookup(*collend, mapping, &x))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3345 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3346 Py_XDECREF(x);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3347 if (x!=Py_None)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3348 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3349 ++collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3350 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3351 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3352 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3353 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3354 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3355 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3356 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3357 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3358 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3359 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3360 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3361 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3362 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3363 known_errorHandler = 0;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3364 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3365 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3366 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3367 raise_translate_exception(&exc, startp, size, collstart-startp, collend-startp, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3368 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3369 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3370 /* No need to check for space, this is a 1:1 replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3371 for (coll = collstart; coll<collend; ++coll)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3372 *str++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3373 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3374 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3375 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3376 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3377 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3378 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3379 for (p = collstart; p < collend; ++p) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3380 char buffer[2+29+1+1];
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3381 char *cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3382 sprintf(buffer, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3383 if (charmaptranslate_makespace(&res, &str, &ressize,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3384 (str-PyUnicode_AS_UNICODE(res))+strlen(buffer)+(endp-collend)))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3385 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3386 for (cp = buffer; *cp; ++cp)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3387 str++ = cp;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3388 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3389 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3390 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3391 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3392 repunicode = unicode_translate_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3393 reason, startp, size, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3394 collstart-startp, collend-startp, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3395 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3396 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3397 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3398 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3399 if (charmaptranslate_makespace(&res, &str, &ressize,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3400 (str-PyUnicode_AS_UNICODE(res))+repsize+(endp-collend))) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3401 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3402 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3403 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3404 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3405 str++ = uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3406 p = startp + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3407 Py_DECREF(repunicode);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3408 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3409 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3410 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3411 /* Resize if we allocated to much */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3412 respos = str-PyUnicode_AS_UNICODE(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3413 if (respos<ressize) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3414 if (_PyUnicode_Resize(&res, respos))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3415 goto onError;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3416 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3417 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3418 Py_XDECREF(errorHandler);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3419 return res;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3420
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3421 onError:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3422 Py_XDECREF(res);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3423 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3424 Py_XDECREF(errorHandler);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3425 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3426 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3427
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3428 PyObject PyUnicode_Translate(PyObject str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3429 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3430 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3431 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3432 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3433
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3434 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3435 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3436 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3437 result = PyUnicode_TranslateCharmap(PyUnicode_AS_UNICODE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3438 PyUnicode_GET_SIZE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3439 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3440 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3441 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3442 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3443
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3444 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3445 Py_XDECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3446 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3447 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3448
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3449 /* --- Decimal Encoder ---------------------------------------------------- */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3450
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3451 int PyUnicode_EncodeDecimal(Py_UNICODE *s,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3452 int length,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3453 char *output,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3454 const char *errors)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3455 {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3456 Py_UNICODE p, end;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3457 PyObject *errorHandler = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3458 PyObject *exc = NULL;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3459 const char *encoding = "decimal";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3460 const char *reason = "invalid decimal Unicode string";
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3461 /* the following variable is used for caching string comparisons
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3462 * -1=not initialized, 0=unknown, 1=strict, 2=replace, 3=ignore, 4=xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3463 int known_errorHandler = -1;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3464
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3465 if (output == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3466 PyErr_BadArgument();
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3467 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3468 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3469
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3470 p = s;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3471 end = s + length;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3472 while (p < end) {
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3473 register Py_UNICODE ch = *p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3474 int decimal;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3475 PyObject *repunicode;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3476 int repsize;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3477 int newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3478 Py_UNICODE *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3479 Py_UNICODE *collstart;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3480 Py_UNICODE *collend;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3481
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3482 if (Py_UNICODE_ISSPACE(ch)) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3483 *output++ = ' ';
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3484 ++p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3485 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3486 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3487 decimal = Py_UNICODE_TODECIMAL(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3488 if (decimal >= 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3489 *output++ = '0' + decimal;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3490 ++p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3491 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3492 }
12275 cbfa95a6a03e Conrad Huang points out that "if (0 < ch < 256)", while legal C, Guido van Rossum <guido@python.org> parents: 12249 diff changeset	3493 if (0 < ch && ch < 256) {
12577 85f00f8cd674 Fix warning detected by VC++ on assignment of Py_UNICODE to char. Guido van Rossum <guido@python.org> parents: 12576 diff changeset	3494 *output++ = (char)ch;
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3495 ++p;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3496 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3497 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3498 /* All other characters are considered unencodable */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3499 collstart = p;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3500 collend = p+1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3501 while (collend < end) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3502 if ((0 < collend && collend < 256) \|\|
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3503 !Py_UNICODE_ISSPACE(*collend) \|\|
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3504 Py_UNICODE_TODECIMAL(*collend))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3505 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3506 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3507 /* cache callback name lookup
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3508 * (if not done yet, i.e. it's the first error) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3509 if (known_errorHandler==-1) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3510 if ((errors==NULL) \|\| (!strcmp(errors, "strict")))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3511 known_errorHandler = 1;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3512 else if (!strcmp(errors, "replace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3513 known_errorHandler = 2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3514 else if (!strcmp(errors, "ignore"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3515 known_errorHandler = 3;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3516 else if (!strcmp(errors, "xmlcharrefreplace"))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3517 known_errorHandler = 4;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3518 else
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3519 known_errorHandler = 0;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3520 }
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3521 switch (known_errorHandler) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3522 case 1: /* strict */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3523 raise_encode_exception(&exc, encoding, s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3524 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3525 case 2: /* replace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3526 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3527 *output++ = '?';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3528 /* fall through */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3529 case 3: /* ignore */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3530 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3531 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3532 case 4: /* xmlcharrefreplace */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3533 /* generate replacement (temporarily (mis)uses p) */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3534 for (p = collstart; p < collend; ++p)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3535 output += sprintf(output, "&#%d;", (int)*p);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3536 p = collend;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3537 break;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3538 default:
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3539 repunicode = unicode_encode_call_errorhandler(errors, &errorHandler,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3540 encoding, reason, s, length, &exc,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3541 collstart-s, collend-s, &newpos);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3542 if (repunicode == NULL)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3543 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3544 /* generate replacement */
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3545 repsize = PyUnicode_GET_SIZE(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3546 for (uni2 = PyUnicode_AS_UNICODE(repunicode); repsize-->0; ++uni2) {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3547 Py_UNICODE ch = *uni2;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3548 if (Py_UNICODE_ISSPACE(ch))
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3549 *output++ = ' ';
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3550 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3551 decimal = Py_UNICODE_TODECIMAL(ch);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3552 if (decimal >= 0)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3553 *output++ = '0' + decimal;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3554 else if (0 < ch && ch < 256)
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3555 *output++ = (char)ch;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3556 else {
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3557 Py_DECREF(repunicode);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3558 raise_encode_exception(&exc, encoding,
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3559 s, length, collstart-s, collend-s, reason);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3560 goto onError;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3561 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3562 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3563 }
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3564 p = s + newpos;
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3565 Py_DECREF(repunicode);
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3566 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3567 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3568 /* 0-terminate the output string */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3569 *output++ = '\0';
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3570 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3571 Py_XDECREF(errorHandler);
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3572 return 0;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3573
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3574 onError:
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3575 Py_XDECREF(exc);
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	3576 Py_XDECREF(errorHandler);
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3577 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3578 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3579
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3580 /* --- Helpers ------------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3581
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3582 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3583 int count(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3584 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3585 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3586 PyUnicodeObject *substring)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3587 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3588 int count = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3589
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3590 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3591 start += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3592 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3593 start = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3594 if (end > self->length)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3595 end = self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3596 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3597 end += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3598 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3599 end = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	3600
12957 97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	3601 if (substring->length == 0)
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	3602 return (end - start + 1);
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	3603
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3604 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3605
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3606 while (start <= end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3607 if (Py_UNICODE_MATCH(self, start, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3608 count++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3609 start += substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3610 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3611 start++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3613 return count;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3614 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3615
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3616 int PyUnicode_Count(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3617 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3618 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3619 int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3620 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3621 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3622
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3623 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3624 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3625 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3626 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3627 if (substr == NULL) {
12957 97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	3628 Py_DECREF(str);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3629 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3630 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3631
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3632 result = count((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3633 start, end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3634 (PyUnicodeObject *)substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3635
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3636 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3637 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3638 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3639 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3640
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3641 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3642 int findstring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3643 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3644 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3645 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3646 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3647 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3648 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3649 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3650 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3651 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3652
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3653 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3654 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3655 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3656 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3657 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3658 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3659
24918 0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith(). Guido van Rossum <guido@python.org> parents: 24831 diff changeset	3660 if (substring->length == 0)
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith(). Guido van Rossum <guido@python.org> parents: 24831 diff changeset	3661 return (direction > 0) ? start : end;
0d5d8e1cdc22 Fix some endcase bugs in unicode rfind()/rindex() and endswith(). Guido van Rossum <guido@python.org> parents: 24831 diff changeset	3662
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3663 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3664
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3665 if (direction < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3666 for (; end >= start; end--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3667 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3668 return end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3669 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3670 for (; start <= end; start++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3671 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3672 return start;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3673 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3674
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3675 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3676 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3677
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3678 int PyUnicode_Find(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3679 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3680 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3681 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3682 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3683 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3684 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3685
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3686 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3687 if (str == NULL)
23561 c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error Marc-André Lemburg <mal@egenix.com> parents: 23542 diff changeset	3688 return -2;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3689 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3690 if (substr == NULL) {
23562 eaca627f2a54 Fix a possible segfault. Found be Neal Norvitz. Marc-André Lemburg <mal@egenix.com> parents: 23561 diff changeset	3691 Py_DECREF(str);
23561 c5e8f89fd358 Fix for bug [ 561796 ] string.find causes lazy error Marc-André Lemburg <mal@egenix.com> parents: 23542 diff changeset	3692 return -2;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3693 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3694
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3695 result = findstring((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3696 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3697 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3698 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3699 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3700 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3701 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3702
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3703 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3704 int tailmatch(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3705 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3706 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3707 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3708 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3709 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3710 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3711 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3712 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3713 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3714
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3715 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3716 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3717
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3718 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3719 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3720 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3721 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3722 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3723 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3724
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3725 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3726 if (end < start)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3727 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3728
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3729 if (direction > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3730 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3731 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3732 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3733 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3734 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3735 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3736
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3737 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3738 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3739
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3740 int PyUnicode_Tailmatch(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3741 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3742 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3743 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3744 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3745 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3746 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3747
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3748 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3749 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3750 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3751 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3752 if (substr == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3753 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3754 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3755 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3756
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3757 result = tailmatch((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3758 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3759 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3760 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3761 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3762 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3763 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3764
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3765 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3766 const Py_UNICODE findchar(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3767 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3768 Py_UNICODE ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3769 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3770 /* like wcschr, but doesn't stop at NULL characters */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3771
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3772 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3773 if (*s == ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3774 return s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3775 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3776 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3777
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3778 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3779 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3780
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3781 /* Apply fixfct filter to the Unicode object self and return a
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3782 reference to the modified object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3783
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3784 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3785 PyObject fixup(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3786 int (fixfct)(PyUnicodeObject s))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3787 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3788
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3789 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3790
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3791 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3792 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3793 return NULL;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3794
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3795 Py_UNICODE_COPY(u->str, self->str, self->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3796
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3797 if (!fixfct(u) && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3798 /* fixfct should return TRUE if it modified the buffer. If
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3799 FALSE, return a reference to the original buffer instead
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3800 (to save space, not time) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3801 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3802 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3803 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3804 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3805 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3806 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3807
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3808 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3809 int fixupper(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3810 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3811 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3812 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3813 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3814
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3815 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3816 register Py_UNICODE ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3817
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3818 ch = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3819 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3820 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3821 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3822 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3823 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3824 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3825
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3826 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3827 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3828
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3829 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3830 int fixlower(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3831 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3832 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3833 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3834 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3835
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3836 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3837 register Py_UNICODE ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3838
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3839 ch = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3840 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3841 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3842 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3843 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3844 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3845 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3846
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3847 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3848 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3849
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3850 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3851 int fixswapcase(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3852 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3853 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3854 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3855 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3856
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3857 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3858 if (Py_UNICODE_ISUPPER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3859 s = Py_UNICODE_TOLOWER(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3860 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3861 } else if (Py_UNICODE_ISLOWER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3862 s = Py_UNICODE_TOUPPER(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3863 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3864 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3865 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3866 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3867
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3868 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3869 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3870
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3871 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3872 int fixcapitalize(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3873 {
16332 bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3874 int len = self->length;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3875 Py_UNICODE *s = self->str;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3876 int status = 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3877
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3878 if (len == 0)
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3879 return 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3880 if (Py_UNICODE_ISLOWER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3881 s = Py_UNICODE_TOUPPER(s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3882 status = 1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3883 }
16332 bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3884 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3885 while (--len > 0) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3886 if (Py_UNICODE_ISUPPER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3887 s = Py_UNICODE_TOLOWER(s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3888 status = 1;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3889 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3890 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3891 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3892 return status;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3893 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3894
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3895 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3896 int fixtitle(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3897 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3898 register Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3899 register Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3900 int previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3901
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3902 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3903 if (PyUnicode_GET_SIZE(self) == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3904 Py_UNICODE ch = Py_UNICODE_TOTITLE(*p);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3905 if (*p != ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3906 *p = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3907 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3908 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3909 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3910 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3911 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3912
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3913 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3914 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3915 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3916 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3917
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3918 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3919 *p = Py_UNICODE_TOLOWER(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3920 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3921 *p = Py_UNICODE_TOTITLE(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3922
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3923 if (Py_UNICODE_ISLOWER(ch) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3924 Py_UNICODE_ISUPPER(ch) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3925 Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3926 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3927 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3928 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3929 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3930 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3931 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3932
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3933 PyObject PyUnicode_Join(PyObject separator,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3934 PyObject *seq)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3935 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3936 Py_UNICODE *sep;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3937 int seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3938 PyUnicodeObject *res = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3939 int reslen = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3940 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3941 int sz = 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3942 int i;
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3943 PyObject *it;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3944
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3945 it = PyObject_GetIter(seq);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3946 if (it == NULL)
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3947 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3948
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3949 if (separator == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3950 Py_UNICODE blank = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3951 sep = &blank;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3952 seplen = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3953 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3954 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3955 separator = PyUnicode_FromObject(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3956 if (separator == NULL)
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3957 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3958 sep = PyUnicode_AS_UNICODE(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3959 seplen = PyUnicode_GET_SIZE(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3960 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3961
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3962 res = _PyUnicode_New(sz);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3963 if (res == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3964 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3965 p = PyUnicode_AS_UNICODE(res);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3966 reslen = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3967
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3968 for (i = 0; ; ++i) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3969 int itemlen;
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3970 PyObject *item = PyIter_Next(it);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3971 if (item == NULL) {
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3972 if (PyErr_Occurred())
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3973 goto onError;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3974 break;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3975 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3976 if (!PyUnicode_Check(item)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3977 PyObject *v;
20007 56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3978 if (!PyString_Check(item)) {
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3979 PyErr_Format(PyExc_TypeError,
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3980 "sequence item %i: expected string or Unicode,"
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3981 " %.80s found",
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3982 i, item->ob_type->tp_name);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3983 Py_DECREF(item);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3984 goto onError;
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3985 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3986 v = PyUnicode_FromObject(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3987 Py_DECREF(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3988 item = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3989 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3990 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3991 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3992 itemlen = PyUnicode_GET_SIZE(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3993 while (reslen + itemlen + seplen >= sz) {
20007 56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3994 if (_PyUnicode_Resize(&res, sz*2)) {
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3995 Py_DECREF(item);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3996 goto onError;
20007 56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3997 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3998 sz *= 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3999 p = PyUnicode_AS_UNICODE(res) + reslen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4000 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4001 if (i > 0) {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4002 Py_UNICODE_COPY(p, sep, seplen);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4003 p += seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4004 reslen += seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4005 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4006 Py_UNICODE_COPY(p, PyUnicode_AS_UNICODE(item), itemlen);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4007 p += itemlen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4008 reslen += itemlen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4009 Py_DECREF(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4010 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4011 if (_PyUnicode_Resize(&res, reslen))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4012 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4013
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4014 Py_XDECREF(separator);
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	4015 Py_DECREF(it);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4016 return (PyObject *)res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4017
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4018 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4019 Py_XDECREF(separator);
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	4020 Py_XDECREF(res);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	4021 Py_DECREF(it);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4022 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4023 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4025 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4026 PyUnicodeObject pad(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4027 int left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4028 int right,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4029 Py_UNICODE fill)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4030 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4031 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4032
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4033 if (left < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4034 left = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4035 if (right < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4036 right = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4037
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4038 if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4039 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4040 return self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4041 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4042
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4043 u = _PyUnicode_New(left + self->length + right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4044 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4045 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4046 Py_UNICODE_FILL(u->str, fill, left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4047 Py_UNICODE_COPY(u->str + left, self->str, self->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4048 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4049 Py_UNICODE_FILL(u->str + left + self->length, fill, right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4050 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4051
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4052 return u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4053 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4054
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4055 #define SPLIT_APPEND(data, left, right) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4056 str = PyUnicode_FromUnicode(data + left, right - left); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4057 if (!str) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4058 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4059 if (PyList_Append(list, str)) { \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4060 Py_DECREF(str); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4061 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4062 } \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4063 else \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4064 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4065
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4066 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4067 PyObject split_whitespace(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4068 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4069 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4070 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4071 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4072 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4073 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4074 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4075
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4076 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4077 /* find a token */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4078 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4079 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4080 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4081 while (i < len && !Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4082 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4083 if (j < i) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4084 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4085 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4086 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4087 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4088 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4089 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4090 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4091 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4092 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4093 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4094 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4095 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4096
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4097 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4098 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4099 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4100 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4101
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4102 PyObject PyUnicode_Splitlines(PyObject string,
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4103 int keepends)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4104 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4105 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4106 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4107 int len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4108 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4109 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4110 Py_UNICODE *data;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4111
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4112 string = PyUnicode_FromObject(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4113 if (string == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4114 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4115 data = PyUnicode_AS_UNICODE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4116 len = PyUnicode_GET_SIZE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4117
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4118 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4119 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4120 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4121
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4122 for (i = j = 0; i < len; ) {
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4123 int eol;
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4124
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4125 /* Find a line and append it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4126 while (i < len && !Py_UNICODE_ISLINEBREAK(data[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4127 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4128
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4129 /* Skip the line break reading CRLF as one line break */
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4130 eol = i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4131 if (i < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4132 if (data[i] == '\r' && i + 1 < len &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4133 data[i+1] == '\n')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4134 i += 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4135 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4136 i++;
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4137 if (keepends)
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4138 eol = i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4139 }
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4140 SPLIT_APPEND(data, j, eol);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4141 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4142 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4143 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4144 SPLIT_APPEND(data, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4145 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4146
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4147 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4148 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4149
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4150 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4151 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4152 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4153 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4154 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4155
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4156 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4157 PyObject split_char(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4158 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4159 Py_UNICODE ch,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4160 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4161 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4162 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4163 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4164 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4165 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4166
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4167 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4168 if (self->str[i] == ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4169 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4170 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4171 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4172 i = j = i + 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4173 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4174 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4175 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4176 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4177 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4178 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4179 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4180
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4181 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4182 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4183 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4184 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4185
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4186 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4187 PyObject split_substring(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4188 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4189 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4190 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4191 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4192 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4193 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4194 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4195 int sublen = substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4196 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4197
15609 0906c8e2c961 Fix off-by-one error in split_substring(). Fixes SF bug #122162. Guido van Rossum <guido@python.org> parents: 15579 diff changeset	4198 for (i = j = 0; i <= len - sublen; ) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4199 if (Py_UNICODE_MATCH(self, i, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4200 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4201 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4202 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4203 i = j = i + sublen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4204 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4205 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4206 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4207 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4208 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4209 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4210 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4211
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4212 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4213 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4214 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4215 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4216
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4217 #undef SPLIT_APPEND
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4218
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4219 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4220 PyObject split(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4221 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4222 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4223 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4224 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4226 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4227 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4228
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4229 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4230 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4231 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4232
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4233 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4234 return split_whitespace(self,list,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4235
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4236 else if (substring->length == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4237 return split_char(self,list,substring->str[0],maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4238
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4239 else if (substring->length == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4240 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4241 PyErr_SetString(PyExc_ValueError, "empty separator");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4242 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4243 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4244 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4245 return split_substring(self,list,substring,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4246 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4247
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4248 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4249 PyObject replace(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4250 PyUnicodeObject *str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4251 PyUnicodeObject *str2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4252 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4253 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4254 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4255
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4256 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4257 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4258
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4259 if (str1->length == 1 && str2->length == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4260 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4261
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4262 /* replace characters */
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4263 if (!findchar(self->str, self->length, str1->str[0]) &&
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4264 PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4265 /* nothing to replace, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4266 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4267 u = self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4268 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4269 Py_UNICODE u1 = str1->str[0];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4270 Py_UNICODE u2 = str2->str[0];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4271
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4272 u = (PyUnicodeObject*) PyUnicode_FromUnicode(
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4273 NULL,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4274 self->length
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4275 );
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4276 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4277 Py_UNICODE_COPY(u->str, self->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4278 self->length);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4279 for (i = 0; i < u->length; i++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4280 if (u->str[i] == u1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4281 if (--maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4282 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4283 u->str[i] = u2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4284 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4285 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	4286 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4287
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4288 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4289 int n, i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4290 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4291
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4292 /* replace strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4293 n = count(self, 0, self->length, str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4294 if (n > maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4295 n = maxcount;
25005 12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4296 if (n == 0) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4297 /* nothing to replace, return original string */
25005 12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4298 if (PyUnicode_CheckExact(self)) {
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4299 Py_INCREF(self);
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4300 u = self;
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4301 }
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4302 else {
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4303 u = (PyUnicodeObject *)
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4304 PyUnicode_FromUnicode(self->str, self->length);
12bb6f0c60f8 Fix SF bug 599128, submitted by Inyeol Lee: .replace() would do the Guido van Rossum <guido@python.org> parents: 25004 diff changeset	4305 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4306 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4307 u = _PyUnicode_New(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4308 self->length + n * (str2->length - str1->length));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4309 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4310 i = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4311 p = u->str;
25004 762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4312 if (str1->length > 0) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4313 while (i <= self->length - str1->length)
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4314 if (Py_UNICODE_MATCH(self, i, str1)) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4315 /* replace string segment */
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4316 Py_UNICODE_COPY(p, str2->str, str2->length);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4317 p += str2->length;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4318 i += str1->length;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4319 if (--n <= 0) {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4320 /* copy remaining part */
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4321 Py_UNICODE_COPY(p, self->str+i, self->length-i);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4322 break;
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4323 }
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4324 } else
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4325 *p++ = self->str[i++];
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4326 } else {
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4327 while (n > 0) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4328 Py_UNICODE_COPY(p, str2->str, str2->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4329 p += str2->length;
25004 762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4330 if (--n <= 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4331 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4332 *p++ = self->str[i++];
25004 762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4333 }
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4334 Py_UNICODE_COPY(p, self->str+i, self->length-i);
762dd09edb83 Code by Inyeol Lee, submitted to SF bug 595350, to implement Guido van Rossum <guido@python.org> parents: 24918 diff changeset	4335 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4336 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4337 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4338 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4340 return (PyObject *) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4341 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4342
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4343 /* --- Unicode Object Methods --------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4344
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4345 PyDoc_STRVAR(title__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4346 "S.title() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4347 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4348 Return a titlecased version of S, i.e. words start with title case\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4349 characters, all remaining cased characters have lower case.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4350
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4351 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4352 unicode_title(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4353 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4354 return fixup(self, fixtitle);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4355 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4356
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4357 PyDoc_STRVAR(capitalize__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4358 "S.capitalize() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4359 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4360 Return a capitalized version of S, i.e. make the first character\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4361 have upper case.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4362
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4363 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4364 unicode_capitalize(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4365 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4366 return fixup(self, fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4367 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4368
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4369 #if 0
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4370 PyDoc_STRVAR(capwords__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4371 "S.capwords() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4372 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4373 Apply .capitalize() to all words in S and return the result with\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4374 normalized whitespace (all whitespace strings are replaced by ' ').");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4375
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4376 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4377 unicode_capwords(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4378 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4379 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4380 PyObject *item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4381 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4382
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4383 /* Split into words */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4384 list = split(self, NULL, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4385 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4386 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4387
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4388 /* Capitalize each word */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4389 for (i = 0; i < PyList_GET_SIZE(list); i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4390 item = fixup((PyUnicodeObject *)PyList_GET_ITEM(list, i),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4391 fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4392 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4393 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4394 Py_DECREF(PyList_GET_ITEM(list, i));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4395 PyList_SET_ITEM(list, i, item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4396 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4397
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4398 /* Join the words to form a new string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4399 item = PyUnicode_Join(NULL, list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4400
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4401 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4402 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4403 return (PyObject *)item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4404 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4405 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4406
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4407 PyDoc_STRVAR(center__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4408 "S.center(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4409 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4410 Return S centered in a Unicode string of length width. Padding is done\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4411 using spaces.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4412
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4413 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4414 unicode_center(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4415 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4416 int marg, left;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4417 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4418
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4419 if (!PyArg_ParseTuple(args, "i:center", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4420 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4421
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4422 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4423 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4424 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4425 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4426
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4427 marg = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4428 left = marg / 2 + (marg & width & 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4429
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4430 return (PyObject*) pad(self, left, marg - left, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4431 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4432
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4433 #if 0
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4434
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4435 /* This code should go into some future Unicode collation support
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4436 module. The basic comparison should compare ordinals on a naive
13946 7a99450da2d6 Clean up warning from Monterey compiler. Trent Mick <trentm@activestate.com> parents: 13890 diff changeset	4437 basis (this is what Java does and thus JPython too). */
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4438
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4439 /* speedy UTF-16 code point order comparison */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4440 /* gleaned from: */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4441 /* http://www-4.ibm.com/software/developer/library/utf16.html?dwzone=unicode */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4442
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	4443 static short utf16Fixup[32] =
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4444 {
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4445 0, 0, 0, 0, 0, 0, 0, 0,
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4446 0, 0, 0, 0, 0, 0, 0, 0,
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4447 0, 0, 0, 0, 0, 0, 0, 0,
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	4448 0, 0, 0, 0x2000, -0x800, -0x800, -0x800, -0x800
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4449 };
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4450
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4451 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4452 unicode_compare(PyUnicodeObject str1, PyUnicodeObject str2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4453 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4454 int len1, len2;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4455
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4456 Py_UNICODE *s1 = str1->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4457 Py_UNICODE *s2 = str2->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4458
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4459 len1 = str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4460 len2 = str2->length;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4461
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4462 while (len1 > 0 && len2 > 0) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	4463 Py_UNICODE c1, c2;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4464
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4465 c1 = *s1++;
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4466 c2 = *s2++;
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4467
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4468 if (c1 > (1<<11) * 26)
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4469 c1 += utf16Fixup[c1>>11];
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4470 if (c2 > (1<<11) * 26)
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4471 c2 += utf16Fixup[c2>>11];
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4472 /* now c1 and c2 are in UTF-32-compatible order */
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4473
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4474 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4475 return (c1 < c2) ? -1 : 1;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4476
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	4477 len1--; len2--;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4478 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4479
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4480 return (len1 < len2) ? -1 : (len1 != len2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4481 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4482
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4483 #else
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4484
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4485 static int
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4486 unicode_compare(PyUnicodeObject str1, PyUnicodeObject str2)
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4487 {
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4488 register int len1, len2;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4489
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4490 Py_UNICODE *s1 = str1->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4491 Py_UNICODE *s2 = str2->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4492
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4493 len1 = str1->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4494 len2 = str2->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4495
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4496 while (len1 > 0 && len2 > 0) {
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4497 Py_UNICODE c1, c2;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4498
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4499 c1 = *s1++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4500 c2 = *s2++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4501
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4502 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4503 return (c1 < c2) ? -1 : 1;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	4504
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4505 len1--; len2--;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4506 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4507
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4508 return (len1 < len2) ? -1 : (len1 != len2);
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4509 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4510
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4511 #endif
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	4512
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4513 int PyUnicode_Compare(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4514 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4515 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4516 PyUnicodeObject u = NULL, v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4517 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4518
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4519 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4520 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4521 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4522 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4523 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4524 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4525 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4526
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 13576 diff changeset	4527 /* Shortcut for empty or interned objects */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4528 if (v == u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4529 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4530 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4531 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4532 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4533
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4534 result = unicode_compare(u, v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4535
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4536 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4537 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4538 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4539
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4540 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4541 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4542 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4543 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4544 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4545
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4546 int PyUnicode_Contains(PyObject *container,
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4547 PyObject *element)
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4548 {
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4549 PyUnicodeObject u = NULL, v = NULL;
24598 12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4550 int result, size;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4551 register const Py_UNICODE lhs, end, *rhs;
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4552
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4553 /* Coerce the two arguments */
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4554 v = (PyUnicodeObject *)PyUnicode_FromObject(element);
13049 0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	4555 if (v == NULL) {
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	4556 PyErr_SetString(PyExc_TypeError,
24598 12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4557 "'in <string>' requires string as left operand");
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4558 goto onError;
13049 0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	4559 }
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4560 u = (PyUnicodeObject *)PyUnicode_FromObject(container);
25895 509bb0b6dfa3 Fix for bug #626172: crash using unicode latin1 single char Marc-André Lemburg <mal@egenix.com> parents: 25719 diff changeset	4561 if (u == NULL)
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	4562 goto onError;
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4563
24598 12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4564 size = PyUnicode_GET_SIZE(v);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4565 rhs = PyUnicode_AS_UNICODE(v);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4566 lhs = PyUnicode_AS_UNICODE(u);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4567
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4568 result = 0;
24598 12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4569 if (size == 1) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4570 end = lhs + PyUnicode_GET_SIZE(u);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4571 while (lhs < end) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4572 if (lhs++ == rhs) {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4573 result = 1;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4574 break;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4575 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4576 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4577 }
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4578 else {
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4579 end = lhs + (PyUnicode_GET_SIZE(u) - size);
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4580 while (lhs <= end) {
24608 b6707e159909 PyUnicode_Contains(): The memcmp() call didn't take into account the Barry Warsaw <barry@python.org> parents: 24598 diff changeset	4581 if (memcmp(lhs++, rhs, size * sizeof(Py_UNICODE)) == 0) {
24598 12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4582 result = 1;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4583 break;
12e146171f3d Committing patch #591250 which provides "str1 in str2" when str1 is a Barry Warsaw <barry@python.org> parents: 24392 diff changeset	4584 }
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4585 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4586 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4587
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4588 Py_DECREF(u);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4589 Py_DECREF(v);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4590 return result;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4591
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4592 onError:
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4593 Py_XDECREF(u);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4594 Py_XDECREF(v);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4595 return -1;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4596 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4597
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4598 /* Concat to string or Unicode object giving a new Unicode object. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4599
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4600 PyObject PyUnicode_Concat(PyObject left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4601 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4602 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4603 PyUnicodeObject u = NULL, v = NULL, *w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4604
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4605 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4606 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4607 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4608 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4609 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4610 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4611 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4613 /* Shortcuts */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4614 if (v == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4615 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4616 return (PyObject *)u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4617 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4618 if (u == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4619 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4620 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4621 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4622
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4623 /* Concat the two Unicode strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4624 w = _PyUnicode_New(u->length + v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4625 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4626 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4627 Py_UNICODE_COPY(w->str, u->str, u->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4628 Py_UNICODE_COPY(w->str + u->length, v->str, v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4629
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4630 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4631 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4632 return (PyObject *)w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4633
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4634 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4635 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4636 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4637 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4638 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4639
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4640 PyDoc_STRVAR(count__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4641 "S.count(sub[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4642 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4643 Return the number of occurrences of substring sub in Unicode string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4644 S[start:end]. Optional arguments start and end are\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4645 interpreted as in slice notation.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4646
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4647 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4648 unicode_count(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4649 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4650 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4651 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4652 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4653 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4654
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4655 if (!PyArg_ParseTuple(args, "O\|O&O&:count", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4656 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4657 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4658
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4659 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4660 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4661 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4662 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4663
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4664 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4665 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4666 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4667 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4668 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4669 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4670 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4671 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4672 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4673 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4674
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4675 result = PyInt_FromLong((long) count(self, start, end, substring));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4676
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4677 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4678 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4679 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4680
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4681 PyDoc_STRVAR(encode__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4682 "S.encode([encoding[,errors]]) -> string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4683 \n\
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	4684 Return an encoded string version of S. Default encoding is the current\n\
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	4685 default string encoding. errors may be given to set a different error\n\
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	4686 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
25091 0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4687 a UnicodeEncodeError. Other possible values are 'ignore', 'replace' and\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4688 'xmlcharrefreplace' as well as any other name registered with\n\
0337dad8403e PEP 293 implemention (from SF patch http://www.python.org/sf/432401) Walter Dörwald <walter@livinglogic.de> parents: 25005 diff changeset	4689 codecs.register_error that can handle UnicodeEncodeErrors.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4690
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4691 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4692 unicode_encode(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4693 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4694 char *encoding = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4695 char *errors = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4696 if (!PyArg_ParseTuple(args, "\|ss:encode", &encoding, &errors))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4697 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4698 return PyUnicode_AsEncodedString((PyObject *)self, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4699 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4700
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4701 PyDoc_STRVAR(expandtabs__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4702 "S.expandtabs([tabsize]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4703 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4704 Return a copy of S where all tab characters are expanded using spaces.\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4705 If tabsize is not given, a tab size of 8 characters is assumed.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4706
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4707 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4708 unicode_expandtabs(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4709 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4710 Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4711 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4712 Py_UNICODE *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4713 int i, j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4714 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4715 int tabsize = 8;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4716
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4717 if (!PyArg_ParseTuple(args, "\|i:expandtabs", &tabsize))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4718 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4719
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 13576 diff changeset	4720 /* First pass: determine size of output string */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4721 i = j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4722 e = self->str + self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4723 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4724 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4725 if (tabsize > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4726 j += tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4727 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4728 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4729 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4730 if (p == '\n' \|\| p == '\r') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4731 i += j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4732 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4733 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4734 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4735
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4736 /* Second pass: create output string and fill it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4737 u = _PyUnicode_New(i + j);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4738 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4739 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4740
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4741 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4742 q = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4743
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4744 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4745 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4746 if (tabsize > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4747 i = tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4748 j += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4749 while (i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4750 *q++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4751 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4752 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4753 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4754 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4755 q++ = p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4756 if (p == '\n' \|\| p == '\r')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4757 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4758 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4759
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4760 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4761 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4762
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4763 PyDoc_STRVAR(find__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4764 "S.find(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4765 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4766 Return the lowest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4767 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4768 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4769 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4770 Return -1 on failure.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4771
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4772 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4773 unicode_find(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4774 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4775 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4776 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4777 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4778 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4779
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4780 if (!PyArg_ParseTuple(args, "O\|O&O&:find", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4781 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4782 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4783 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4784 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4785 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4786 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4787
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4788 result = PyInt_FromLong(findstring(self, substring, start, end, 1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4789
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4790 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4791 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4792 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4793
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4794 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4795 unicode_getitem(PyUnicodeObject *self, int index)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4796 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4797 if (index < 0 \|\| index >= self->length) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4798 PyErr_SetString(PyExc_IndexError, "string index out of range");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4799 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4800 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4801
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4802 return (PyObject*) PyUnicode_FromUnicode(&self->str[index], 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4803 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4804
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4805 static long
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4806 unicode_hash(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4807 {
13522 acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4808 /* Since Unicode objects compare equal to their ASCII string
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4809 counterparts, they should use the individual character values
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4810 as basis for their hash value. This is needed to assure that
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4811 strings and Unicode objects behave in the same way as
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4812 dictionary keys. */
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4813
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4814 register int len;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4815 register Py_UNICODE *p;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4816 register long x;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4817
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4818 if (self->hash != -1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4819 return self->hash;
13522 acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4820 len = PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4821 p = PyUnicode_AS_UNICODE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4822 x = *p << 7;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4823 while (--len >= 0)
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4824 x = (1000003x) ^ p++;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4825 x ^= PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4826 if (x == -1)
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4827 x = -2;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4828 self->hash = x;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4829 return x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4830 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4831
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4832 PyDoc_STRVAR(index__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4833 "S.index(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4834 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4835 Like S.find() but raise ValueError when the substring is not found.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4836
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4837 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4838 unicode_index(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4839 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4840 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4841 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4842 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4843 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4844
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4845 if (!PyArg_ParseTuple(args, "O\|O&O&:index", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4846 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4847 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4848
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4849 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4850 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4851 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4852 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4853
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4854 result = findstring(self, substring, start, end, 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4855
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4856 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4857 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4858 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4859 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4860 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4861 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4862 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4863
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4864 PyDoc_STRVAR(islower__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4865 "S.islower() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4866 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4867 Return True if all cased characters in S are lowercase and there is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4868 at least one cased character in S, False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4869
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4870 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4871 unicode_islower(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4872 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4873 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4874 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4875 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4876
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4877 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4878 if (PyUnicode_GET_SIZE(self) == 1)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4879 return PyBool_FromLong(Py_UNICODE_ISLOWER(*p));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4880
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4881 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4882 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4883 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4884
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4885 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4886 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4887 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4888 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4889
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4890 if (Py_UNICODE_ISUPPER(ch) \|\| Py_UNICODE_ISTITLE(ch))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4891 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4892 else if (!cased && Py_UNICODE_ISLOWER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4893 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4894 }
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4895 return PyBool_FromLong(cased);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4896 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4897
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4898 PyDoc_STRVAR(isupper__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4899 "S.isupper() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4900 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4901 Return True if all cased characters in S are uppercase and there is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4902 at least one cased character in S, False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4903
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4904 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4905 unicode_isupper(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4906 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4907 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4908 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4909 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4910
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4911 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4912 if (PyUnicode_GET_SIZE(self) == 1)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4913 return PyBool_FromLong(Py_UNICODE_ISUPPER(*p) != 0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4914
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4915 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4916 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4917 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4918
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4919 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4920 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4921 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4922 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4923
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4924 if (Py_UNICODE_ISLOWER(ch) \|\| Py_UNICODE_ISTITLE(ch))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4925 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4926 else if (!cased && Py_UNICODE_ISUPPER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4927 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4928 }
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4929 return PyBool_FromLong(cased);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4930 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4931
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4932 PyDoc_STRVAR(istitle__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4933 "S.istitle() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4934 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4935 Return True if S is a titlecased string, i.e. upper- and titlecase\n\
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4936 characters may only follow uncased characters and lowercase characters\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4937 only cased ones. Return False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4938
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4939 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4940 unicode_istitle(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4941 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4942 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4943 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4944 int cased, previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4945
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4946 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4947 if (PyUnicode_GET_SIZE(self) == 1)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4948 return PyBool_FromLong((Py_UNICODE_ISTITLE(*p) != 0) \|\|
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4949 (Py_UNICODE_ISUPPER(*p) != 0));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4950
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4951 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4952 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4953 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4954
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4955 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4956 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4957 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4958 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4959 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4960
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4961 if (Py_UNICODE_ISUPPER(ch) \|\| Py_UNICODE_ISTITLE(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4962 if (previous_is_cased)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4963 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4964 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4965 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4966 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4967 else if (Py_UNICODE_ISLOWER(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4968 if (!previous_is_cased)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4969 return PyBool_FromLong(0);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4970 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4971 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4972 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4973 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4974 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4975 }
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4976 return PyBool_FromLong(cased);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4977 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4978
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4979 PyDoc_STRVAR(isspace__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4980 "S.isspace() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4981 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4982 Return True if there are only whitespace characters in S,\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	4983 False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4984
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4985 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4986 unicode_isspace(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4987 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4988 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4989 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4990
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4991 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4992 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4993 Py_UNICODE_ISSPACE(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4994 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4995
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4996 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4997 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	4998 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4999
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5000 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5001 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5002 if (!Py_UNICODE_ISSPACE(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5003 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5004 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5005 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5006 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5007
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5008 PyDoc_STRVAR(isalpha__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5009 "S.isalpha() -> bool\n\
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5010 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5011 Return True if all characters in S are alphabetic\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5012 and there is at least one character in S, False otherwise.");
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5013
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5014 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5015 unicode_isalpha(PyUnicodeObject *self)
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5016 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5017 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5018 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5019
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5020 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5021 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5022 Py_UNICODE_ISALPHA(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5023 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5024
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5025 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5026 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5027 return PyBool_FromLong(0);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5028
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5029 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5030 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5031 if (!Py_UNICODE_ISALPHA(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5032 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5033 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5034 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5035 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5036
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5037 PyDoc_STRVAR(isalnum__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5038 "S.isalnum() -> bool\n\
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5039 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5040 Return True if all characters in S are alphanumeric\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5041 and there is at least one character in S, False otherwise.");
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5042
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5043 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5044 unicode_isalnum(PyUnicodeObject *self)
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5045 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5046 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5047 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5048
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5049 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5050 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5051 Py_UNICODE_ISALNUM(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5052 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5053
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5054 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5055 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5056 return PyBool_FromLong(0);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5057
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5058 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5059 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5060 if (!Py_UNICODE_ISALNUM(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5061 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5062 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5063 return PyBool_FromLong(1);
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5064 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	5065
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5066 PyDoc_STRVAR(isdecimal__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5067 "S.isdecimal() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5068 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5069 Return True if there are only decimal characters in S,\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5070 False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5071
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5072 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5073 unicode_isdecimal(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5074 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5075 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5076 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5077
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5078 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5079 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5080 Py_UNICODE_ISDECIMAL(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5081 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5082
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5083 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5084 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5085 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5086
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5087 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5088 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5089 if (!Py_UNICODE_ISDECIMAL(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5090 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5091 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5092 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5093 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5094
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5095 PyDoc_STRVAR(isdigit__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5096 "S.isdigit() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5097 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5098 Return True if there are only digit characters in S,\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5099 False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5100
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5101 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5102 unicode_isdigit(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5103 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5104 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5105 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5106
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5107 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5108 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5109 Py_UNICODE_ISDIGIT(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5110 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5111
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5112 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5113 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5114 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5115
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5116 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5117 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5118 if (!Py_UNICODE_ISDIGIT(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5119 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5120 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5121 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5122 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5123
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5124 PyDoc_STRVAR(isnumeric__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5125 "S.isnumeric() -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5126 \n\
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5127 Return True if there are only numeric characters in S,\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5128 False otherwise.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5129
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5130 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5131 unicode_isnumeric(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5132 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5133 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5134 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5135
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5136 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5137 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5138 Py_UNICODE_ISNUMERIC(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5139 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5140
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5141 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5142 if (PyString_GET_SIZE(self) == 0)
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5143 return PyBool_FromLong(0);
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	5144
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5145 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5146 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5147 if (!Py_UNICODE_ISNUMERIC(*p))
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5148 return PyBool_FromLong(0);
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5149 }
aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5150 return PyBool_FromLong(1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5151 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5152
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5153 PyDoc_STRVAR(join__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5154 "S.join(sequence) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5155 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5156 Return a string which is the concatenation of the strings in the\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5157 sequence. The separator between elements is S.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5158
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5159 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5160 unicode_join(PyObject self, PyObject data)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5161 {
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5162 return PyUnicode_Join(self, data);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5163 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5164
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5165 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5166 unicode_length(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5167 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5168 return self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5169 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5170
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5171 PyDoc_STRVAR(ljust__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5172 "S.ljust(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5173 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5174 Return S left justified in a Unicode string of length width. Padding is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5175 done using spaces.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5176
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5177 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5178 unicode_ljust(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5179 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5180 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5181 if (!PyArg_ParseTuple(args, "i:ljust", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5182 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5183
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	5184 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5185 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5186 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5187 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5188
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5189 return (PyObject*) pad(self, 0, width - self->length, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5190 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5191
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5192 PyDoc_STRVAR(lower__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5193 "S.lower() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5194 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5195 Return a copy of the string S converted to lowercase.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5196
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5197 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5198 unicode_lower(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5199 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5200 return fixup(self, fixlower);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5202
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5203 #define LEFTSTRIP 0
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5204 #define RIGHTSTRIP 1
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5205 #define BOTHSTRIP 2
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5206
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5207 /* Arrays indexed by above */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5208 static const char *stripformat[] = {"\|O:lstrip", "\|O:rstrip", "\|O:strip"};
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5209
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5210 #define STRIPNAME(i) (stripformat[i]+3)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5211
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5212 static const Py_UNICODE *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5213 unicode_memchr(const Py_UNICODE *s, Py_UNICODE c, size_t n)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5214 {
23228 6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which Tim Peters <tim.peters@gmail.com> parents: 23225 diff changeset	5215 size_t i;
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which Tim Peters <tim.peters@gmail.com> parents: 23225 diff changeset	5216 for (i = 0; i < n; ++i)
6db6e3dc8366 unicode_memchr(): Squashed gratuitous int-vs-size_t mismatch (which Tim Peters <tim.peters@gmail.com> parents: 23225 diff changeset	5217 if (s[i] == c)
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5218 return s+i;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5219 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5220 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5221
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5222 /* externally visible for str.strip(unicode) */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5223 PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5224 _PyUnicode_XStrip(PyUnicodeObject self, int striptype, PyObject sepobj)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5225 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5226 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5227 int len = PyUnicode_GET_SIZE(self);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5228 Py_UNICODE *sep = PyUnicode_AS_UNICODE(sepobj);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5229 int seplen = PyUnicode_GET_SIZE(sepobj);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5230 int i, j;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5231
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5232 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5233 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5234 while (i < len && unicode_memchr(sep, s[i], seplen)) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5235 i++;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5236 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5237 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5238
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5239 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5240 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5241 do {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5242 j--;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5243 } while (j >= i && unicode_memchr(sep, s[j], seplen));
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5244 j++;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5245 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5246
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5247 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5248 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5249 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5250 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5251 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5252 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5253 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5254
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5255
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5256 static PyObject *
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5257 do_strip(PyUnicodeObject *self, int striptype)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5258 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5259 Py_UNICODE *s = PyUnicode_AS_UNICODE(self);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5260 int len = PyUnicode_GET_SIZE(self), i, j;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5261
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5262 i = 0;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5263 if (striptype != RIGHTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5264 while (i < len && Py_UNICODE_ISSPACE(s[i])) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5265 i++;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5266 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5267 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5268
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5269 j = len;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5270 if (striptype != LEFTSTRIP) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5271 do {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5272 j--;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5273 } while (j >= i && Py_UNICODE_ISSPACE(s[j]));
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5274 j++;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5275 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5276
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5277 if (i == 0 && j == len && PyUnicode_CheckExact(self)) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5278 Py_INCREF(self);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5279 return (PyObject*)self;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5280 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5281 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5282 return PyUnicode_FromUnicode(s+i, j-i);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5283 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5284
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5285
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5286 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5287 do_argstrip(PyUnicodeObject self, int striptype, PyObject args)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5288 {
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5289 PyObject *sep = NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5290
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5291 if (!PyArg_ParseTuple(args, (char *)stripformat[striptype], &sep))
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5292 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5293
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5294 if (sep != NULL && sep != Py_None) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5295 if (PyUnicode_Check(sep))
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5296 return _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5297 else if (PyString_Check(sep)) {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5298 PyObject *res;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5299 sep = PyUnicode_FromObject(sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5300 if (sep==NULL)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5301 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5302 res = _PyUnicode_XStrip(self, striptype, sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5303 Py_DECREF(sep);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5304 return res;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5305 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5306 else {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5307 PyErr_Format(PyExc_TypeError,
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5308 "%s arg must be None, unicode or str",
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5309 STRIPNAME(striptype));
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5310 return NULL;
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5311 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5312 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5313
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5314 return do_strip(self, striptype);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5315 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5316
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5317
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5318 PyDoc_STRVAR(strip__doc__,
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5319 "S.strip([chars]) -> unicode\n\
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5320 \n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5321 Return a copy of the string S with leading and trailing\n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5322 whitespace removed.\n\
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5323 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5324 If chars is a str, it will be converted to unicode before stripping");
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5325
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5326 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5327 unicode_strip(PyUnicodeObject self, PyObject args)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5328 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5329 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5330 return do_strip(self, BOTHSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5331 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5332 return do_argstrip(self, BOTHSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5333 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5334
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5335
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5336 PyDoc_STRVAR(lstrip__doc__,
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5337 "S.lstrip([chars]) -> unicode\n\
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5338 \n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5339 Return a copy of the string S with leading whitespace removed.\n\
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5340 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5341 If chars is a str, it will be converted to unicode before stripping");
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5342
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5343 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5344 unicode_lstrip(PyUnicodeObject self, PyObject args)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5345 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5346 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5347 return do_strip(self, LEFTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5348 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5349 return do_argstrip(self, LEFTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5350 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5351
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5352
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5353 PyDoc_STRVAR(rstrip__doc__,
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5354 "S.rstrip([chars]) -> unicode\n\
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5355 \n\
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5356 Return a copy of the string S with trailing whitespace removed.\n\
28575 215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5357 If chars is given and not None, remove characters in chars instead.\n\
215f1f4851a4 Attempt to make all the various string strip methods the same. Neal Norwitz <nnorwitz@gmail.com>* parents: 28559 diff changeset	5358 If chars is a str, it will be converted to unicode before stripping");
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5359
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5360 static PyObject *
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5361 unicode_rstrip(PyUnicodeObject self, PyObject args)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5362 {
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5363 if (PyTuple_GET_SIZE(args) == 0)
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5364 return do_strip(self, RIGHTSTRIP); /* Common case */
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5365 else
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5366 return do_argstrip(self, RIGHTSTRIP, args);
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5367 }
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5368
e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5369
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5370 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5371 unicode_repeat(PyUnicodeObject *str, int len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5372 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5373 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5374 Py_UNICODE *p;
14492 7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5375 int nchars;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5376 size_t nbytes;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5377
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5378 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5379 len = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5380
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	5381 if (len == 1 && PyUnicode_CheckExact(str)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5382 /* no repeat, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5383 Py_INCREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5384 return (PyObject*) str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5385 }
14492 7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5386
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5387 /* ensure # of chars needed doesn't overflow int and # of bytes
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5388 * needed doesn't overflow size_t
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5389 */
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5390 nchars = len * str->length;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5391 if (len && nchars / len != str->length) {
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5392 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5393 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5394 return NULL;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5395 }
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5396 nbytes = (nchars + 1) * sizeof(Py_UNICODE);
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5397 if (nbytes / sizeof(Py_UNICODE) != (size_t)(nchars + 1)) {
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5398 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5399 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5400 return NULL;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5401 }
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	5402 u = _PyUnicode_New(nchars);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5403 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5404 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5405
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5406 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5407
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5408 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5409 Py_UNICODE_COPY(p, str->str, str->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5410 p += str->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5411 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5412
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5413 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5414 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5415
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5416 PyObject PyUnicode_Replace(PyObject obj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5417 PyObject *subobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5418 PyObject *replobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5419 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5420 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5421 PyObject *self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5422 PyObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5423 PyObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5424 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5425
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5426 self = PyUnicode_FromObject(obj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5427 if (self == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5428 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5429 str1 = PyUnicode_FromObject(subobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5430 if (str1 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5431 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5432 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5433 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5434 str2 = PyUnicode_FromObject(replobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5435 if (str2 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5436 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5437 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5438 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5439 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5440 result = replace((PyUnicodeObject *)self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5441 (PyUnicodeObject *)str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5442 (PyUnicodeObject *)str2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5443 maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5444 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5445 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5446 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5447 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5448 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5449
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5450 PyDoc_STRVAR(replace__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5451 "S.replace (old, new[, maxsplit]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5452 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5453 Return a copy of S with all occurrences of substring\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5454 old replaced by new. If the optional argument maxsplit is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5455 given, only the first maxsplit occurrences are replaced.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5456
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5457 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5458 unicode_replace(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5459 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5460 PyUnicodeObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5461 PyUnicodeObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5462 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5463 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5464
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5465 if (!PyArg_ParseTuple(args, "OO\|i:replace", &str1, &str2, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5466 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5467 str1 = (PyUnicodeObject )PyUnicode_FromObject((PyObject )str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5468 if (str1 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5469 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5470 str2 = (PyUnicodeObject )PyUnicode_FromObject((PyObject )str2);
27774 bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	5471 if (str2 == NULL) {
bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	5472 Py_DECREF(str1);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5473 return NULL;
27774 bfc524379307 Fix two refcounting bugs Walter Dörwald <walter@livinglogic.de> parents: 27555 diff changeset	5474 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5475
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5476 result = replace(self, str1, str2, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5477
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5478 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5479 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5480 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5481 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5482
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5483 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5484 PyObject unicode_repr(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5485 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5486 return unicodeescape_string(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5487 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5488 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5489 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5490
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5491 PyDoc_STRVAR(rfind__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5492 "S.rfind(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5493 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5494 Return the highest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5495 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5496 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5497 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5498 Return -1 on failure.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5499
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5500 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5501 unicode_rfind(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5502 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5503 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5504 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5505 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5506 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5507
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5508 if (!PyArg_ParseTuple(args, "O\|O&O&:rfind", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5509 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5510 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5511 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5512 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5513 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5514 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5515
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5516 result = PyInt_FromLong(findstring(self, substring, start, end, -1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5517
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5518 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5519 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5520 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5521
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5522 PyDoc_STRVAR(rindex__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5523 "S.rindex(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5524 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5525 Like S.rfind() but raise ValueError when the substring is not found.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5526
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5527 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5528 unicode_rindex(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5529 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5530 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5531 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5532 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5533 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5534
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5535 if (!PyArg_ParseTuple(args, "O\|O&O&:rindex", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5536 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5537 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5538 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5539 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5540 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5541 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5542
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5543 result = findstring(self, substring, start, end, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5544
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5545 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5546 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5547 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5548 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5549 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5550 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5551 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5552
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5553 PyDoc_STRVAR(rjust__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5554 "S.rjust(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5555 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5556 Return S right justified in a Unicode string of length width. Padding is\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5557 done using spaces.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5558
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5559 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5560 unicode_rjust(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5561 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5562 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5563 if (!PyArg_ParseTuple(args, "i:rjust", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5564 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5565
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	5566 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5567 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5568 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5569 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5570
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5571 return (PyObject*) pad(self, width - self->length, 0, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5572 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5573
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5574 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5575 unicode_slice(PyUnicodeObject *self, int start, int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5576 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5577 /* standard clamping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5578 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5579 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5580 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5581 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5582 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5583 end = self->length;
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	5584 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5585 /* full slice, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5586 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5587 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5588 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5589 if (start > end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5590 start = end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5591 /* copy slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5592 return (PyObject*) PyUnicode_FromUnicode(self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5593 end - start);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5594 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5595
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5596 PyObject PyUnicode_Split(PyObject s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5597 PyObject *sep,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5598 int maxsplit)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5599 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5600 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5601
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5602 s = PyUnicode_FromObject(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5603 if (s == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5604 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5605 if (sep != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5606 sep = PyUnicode_FromObject(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5607 if (sep == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5608 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5609 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5610 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5611 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5613 result = split((PyUnicodeObject )s, (PyUnicodeObject )sep, maxsplit);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5614
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5615 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5616 Py_XDECREF(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5617 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5618 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5619
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5620 PyDoc_STRVAR(split__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5621 "S.split([sep [,maxsplit]]) -> list of strings\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5622 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5623 Return a list of the words in S, using sep as the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5624 delimiter string. If maxsplit is given, at most maxsplit\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5625 splits are done. If sep is not specified, any whitespace string\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5626 is a separator.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5627
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5628 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5629 unicode_split(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5630 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5631 PyObject *substring = Py_None;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5632 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5633
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5634 if (!PyArg_ParseTuple(args, "\|Oi:split", &substring, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5635 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5636
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5637 if (substring == Py_None)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5638 return split(self, NULL, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5639 else if (PyUnicode_Check(substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5640 return split(self, (PyUnicodeObject *)substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5641 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5642 return PyUnicode_Split((PyObject *)self, substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5643 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5644
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5645 PyDoc_STRVAR(splitlines__doc__,
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	5646 "S.splitlines([keepends]]) -> list of strings\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5647 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5648 Return a list of the lines in S, breaking at line boundaries.\n\
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	5649 Line breaks are not included in the resulting list unless keepends\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5650 is given and true.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5651
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5652 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5653 unicode_splitlines(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5654 {
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	5655 int keepends = 0;
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	5656
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	5657 if (!PyArg_ParseTuple(args, "\|i:splitlines", &keepends))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5658 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5659
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	5660 return PyUnicode_Splitlines((PyObject *)self, keepends);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5661 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5662
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5663 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5664 PyObject unicode_str(PyUnicodeObject self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5665 {
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5666 return PyUnicode_AsEncodedString((PyObject *)self, NULL, NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5667 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5668
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5669 PyDoc_STRVAR(swapcase__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5670 "S.swapcase() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5671 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5672 Return a copy of S with uppercase characters converted to lowercase\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5673 and vice versa.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5674
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5675 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5676 unicode_swapcase(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5677 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5678 return fixup(self, fixswapcase);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5679 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5680
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5681 PyDoc_STRVAR(translate__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5682 "S.translate(table) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5683 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5684 Return a copy of the string S, where all characters have been mapped\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5685 through the given translation table, which must be a mapping of\n\
25120 953ebe0c0c25 Change the unicode.translate docstring to document that Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	5686 Unicode ordinals to Unicode ordinals, Unicode strings or None.\n\
953ebe0c0c25 Change the unicode.translate docstring to document that Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	5687 Unmapped characters are left untouched. Characters mapped to None\n\
953ebe0c0c25 Change the unicode.translate docstring to document that Walter Dörwald <walter@livinglogic.de> parents: 25091 diff changeset	5688 are deleted.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5689
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5690 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5691 unicode_translate(PyUnicodeObject self, PyObject table)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5692 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5693 return PyUnicode_TranslateCharmap(self->str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5694 self->length,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5695 table,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5696 "ignore");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5697 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5698
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5699 PyDoc_STRVAR(upper__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5700 "S.upper() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5701 \n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5702 Return a copy of S converted to uppercase.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5703
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5704 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5705 unicode_upper(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5706 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5707 return fixup(self, fixupper);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5708 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5709
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5710 PyDoc_STRVAR(zfill__doc__,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5711 "S.zfill(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5712 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5713 Pad a numeric string x with zeros on the left, to fill a field\n\
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5714 of the specified width. The string x is never truncated.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5715
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5716 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5717 unicode_zfill(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5718 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5719 int fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5720 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5721
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5722 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5723 if (!PyArg_ParseTuple(args, "i:zfill", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5724 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5725
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5726 if (self->length >= width) {
23051 19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5727 if (PyUnicode_CheckExact(self)) {
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5728 Py_INCREF(self);
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5729 return (PyObject*) self;
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5730 }
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5731 else
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5732 return PyUnicode_FromUnicode(
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5733 PyUnicode_AS_UNICODE(self),
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5734 PyUnicode_GET_SIZE(self)
19ea00feda33 Return the orginal string only if it's a real str or unicode Walter Dörwald <walter@livinglogic.de> parents: 23045 diff changeset	5735 );
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5736 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5737
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5738 fill = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5739
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5740 u = pad(self, fill, 0, '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5741
23045 8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	5742 if (u == NULL)
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	5743 return NULL;
8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	5744
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5745 if (u->str[fill] == '+' \|\| u->str[fill] == '-') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5746 /* move sign to beginning of string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5747 u->str[0] = u->str[fill];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5748 u->str[fill] = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5749 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5750
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5751 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5752 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5753
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5754 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5755 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5756 unicode_freelistsize(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5757 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5758 return PyInt_FromLong(unicode_freelist_size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5759 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5760 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5761
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5762 PyDoc_STRVAR(startswith__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5763 "S.startswith(prefix[, start[, end]]) -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5764 \n\
28559 aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	5765 Return True if S starts with the specified prefix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	5766 With optional start, test S beginning at that position.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	5767 With optional end, stop comparing S at that position.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5768
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5769 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5770 unicode_startswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5771 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5772 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5773 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5774 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5775 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5776 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5777
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5778 if (!PyArg_ParseTuple(args, "O\|O&O&:startswith", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5779 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5780 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5781 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5782 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5783 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5784 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5785
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5786 result = PyBool_FromLong(tailmatch(self, substring, start, end, -1));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5787
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5788 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5789 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5790 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5791
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5792
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	5793 PyDoc_STRVAR(endswith__doc__,
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5794 "S.endswith(suffix[, start[, end]]) -> bool\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5795 \n\
28559 aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	5796 Return True if S ends with the specified suffix, False otherwise.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	5797 With optional start, test S beginning at that position.\n\
aff080f2efb4 Reformat a few docstrings that caused line wraps in help() output. Guido van Rossum <guido@python.org> parents: 28494 diff changeset	5798 With optional end, stop comparing S at that position.");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5799
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5800 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5801 unicode_endswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5802 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5803 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5804 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5805 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5806 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5807 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5808
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5809 if (!PyArg_ParseTuple(args, "O\|O&O&:endswith", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	5810 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5811 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5812 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5813 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5814 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5815 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5816
22785 aa8d59823add Add the 'bool' type and its values 'False' and 'True', as described in Guido van Rossum <guido@python.org> parents: 22521 diff changeset	5817 result = PyBool_FromLong(tailmatch(self, substring, start, end, +1));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5818
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5819 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5820 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5821 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5822
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5823
27521 17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5824
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5825 static PyObject *
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5826 unicode_getnewargs(PyUnicodeObject *v)
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5827 {
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5828 return Py_BuildValue("(u#)", v->str, v->length);
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5829 }
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5830
17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5831
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5832 static PyMethodDef unicode_methods[] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5833
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5834 /* Order is according to common usage: often used methods should
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5835 appear first, since lookup is done sequentially. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5836
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5837 {"encode", (PyCFunction) unicode_encode, METH_VARARGS, encode__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5838 {"replace", (PyCFunction) unicode_replace, METH_VARARGS, replace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5839 {"split", (PyCFunction) unicode_split, METH_VARARGS, split__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5840 {"join", (PyCFunction) unicode_join, METH_O, join__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5841 {"capitalize", (PyCFunction) unicode_capitalize, METH_NOARGS, capitalize__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5842 {"title", (PyCFunction) unicode_title, METH_NOARGS, title__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5843 {"center", (PyCFunction) unicode_center, METH_VARARGS, center__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5844 {"count", (PyCFunction) unicode_count, METH_VARARGS, count__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5845 {"expandtabs", (PyCFunction) unicode_expandtabs, METH_VARARGS, expandtabs__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5846 {"find", (PyCFunction) unicode_find, METH_VARARGS, find__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5847 {"index", (PyCFunction) unicode_index, METH_VARARGS, index__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5848 {"ljust", (PyCFunction) unicode_ljust, METH_VARARGS, ljust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5849 {"lower", (PyCFunction) unicode_lower, METH_NOARGS, lower__doc__},
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5850 {"lstrip", (PyCFunction) unicode_lstrip, METH_VARARGS, lstrip__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5851 /* {"maketrans", (PyCFunction) unicode_maketrans, METH_VARARGS, maketrans__doc__}, */
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5852 {"rfind", (PyCFunction) unicode_rfind, METH_VARARGS, rfind__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5853 {"rindex", (PyCFunction) unicode_rindex, METH_VARARGS, rindex__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5854 {"rjust", (PyCFunction) unicode_rjust, METH_VARARGS, rjust__doc__},
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5855 {"rstrip", (PyCFunction) unicode_rstrip, METH_VARARGS, rstrip__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5856 {"splitlines", (PyCFunction) unicode_splitlines, METH_VARARGS, splitlines__doc__},
23225 e639e34ef901 Apply patch diff.txt from SF feature request Walter Dörwald <walter@livinglogic.de> parents: 23205 diff changeset	5857 {"strip", (PyCFunction) unicode_strip, METH_VARARGS, strip__doc__},
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5858 {"swapcase", (PyCFunction) unicode_swapcase, METH_NOARGS, swapcase__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5859 {"translate", (PyCFunction) unicode_translate, METH_O, translate__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5860 {"upper", (PyCFunction) unicode_upper, METH_NOARGS, upper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5861 {"startswith", (PyCFunction) unicode_startswith, METH_VARARGS, startswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5862 {"endswith", (PyCFunction) unicode_endswith, METH_VARARGS, endswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5863 {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5864 {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5865 {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5866 {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5867 {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5868 {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5869 {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5870 {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5871 {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
23045 8b1268bc371a Apply the second version of SF patch http://www.python.org/sf/536241 Walter Dörwald <walter@livinglogic.de> parents: 22981 diff changeset	5872 {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5873 #if 0
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5874 {"capwords", (PyCFunction) unicode_capwords, METH_NOARGS, capwords__doc__},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5875 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5876
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5877 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5878 /* This one is just used for debugging the implementation. */
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	5879 {"freelistsize", (PyCFunction) unicode_freelistsize, METH_NOARGS},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5880 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5881
27521 17bfd5dbc0f1 Implement appropriate __getnewargs__ for all immutable subclassable builtin Guido van Rossum <guido@python.org> parents: 27175 diff changeset	5882 {"__getnewargs__", (PyCFunction)unicode_getnewargs, METH_NOARGS},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5883 {NULL, NULL}
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5884 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5885
26274 958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5886 static PyObject *
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5887 unicode_mod(PyObject v, PyObject w)
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5888 {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5889 if (!PyUnicode_Check(v)) {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5890 Py_INCREF(Py_NotImplemented);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5891 return Py_NotImplemented;
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5892 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5893 return PyUnicode_Format(v, w);
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5894 }
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5895
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5896 static PyNumberMethods unicode_as_number = {
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5897 0, /nb_add/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5898 0, /nb_subtract/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5899 0, /nb_multiply/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5900 0, /nb_divide/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5901 unicode_mod, /nb_remainder/
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5902 };
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	5903
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5904 static PySequenceMethods unicode_as_sequence = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5905 (inquiry) unicode_length, /* sq_length */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5906 (binaryfunc) PyUnicode_Concat, /* sq_concat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5907 (intargfunc) unicode_repeat, /* sq_repeat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5908 (intargfunc) unicode_getitem, /* sq_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5909 (intintargfunc) unicode_slice, /* sq_slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5910 0, /* sq_ass_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5911 0, /* sq_ass_slice */
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	5912 (objobjproc)PyUnicode_Contains, /sq_contains/
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5913 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5914
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5915 static PyObject*
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5916 unicode_subscript(PyUnicodeObject* self, PyObject* item)
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5917 {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5918 if (PyInt_Check(item)) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5919 long i = PyInt_AS_LONG(item);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5920 if (i < 0)
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5921 i += PyString_GET_SIZE(self);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5922 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5923 } else if (PyLong_Check(item)) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5924 long i = PyLong_AsLong(item);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5925 if (i == -1 && PyErr_Occurred())
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5926 return NULL;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5927 if (i < 0)
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5928 i += PyString_GET_SIZE(self);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5929 return unicode_getitem(self, i);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5930 } else if (PySlice_Check(item)) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5931 int start, stop, step, slicelength, cur, i;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5932 Py_UNICODE* source_buf;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5933 Py_UNICODE* result_buf;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5934 PyObject* result;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5935
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5936 if (PySlice_GetIndicesEx((PySliceObject*)item, PyString_GET_SIZE(self),
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5937 &start, &stop, &step, &slicelength) < 0) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5938 return NULL;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5939 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5940
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5941 if (slicelength <= 0) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5942 return PyUnicode_FromUnicode(NULL, 0);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5943 } else {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5944 source_buf = PyUnicode_AS_UNICODE((PyObject*)self);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5945 result_buf = PyMem_MALLOC(slicelength*sizeof(Py_UNICODE));
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5946
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5947 for (cur = start, i = 0; i < slicelength; cur += step, i++) {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5948 result_buf[i] = source_buf[cur];
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5949 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5950
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5951 result = PyUnicode_FromUnicode(result_buf, slicelength);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5952 PyMem_FREE(result_buf);
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5953 return result;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5954 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5955 } else {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5956 PyErr_SetString(PyExc_TypeError, "string indices must be integers");
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5957 return NULL;
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5958 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5959 }
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5960
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5961 static PyMappingMethods unicode_as_mapping = {
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5962 (inquiry)unicode_length, /* mp_length */
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5963 (binaryfunc)unicode_subscript, /* mp_subscript */
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5964 (objobjargproc)0, /* mp_ass_subscript */
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5965 };
245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	5966
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5967 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5968 unicode_buffer_getreadbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5969 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5970 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5971 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5972 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5973 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5974 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5975 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5976 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5977 ptr = (void ) self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5978 return PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5979 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5980
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5981 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5982 unicode_buffer_getwritebuf(PyUnicodeObject *self, int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5983 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5984 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5985 PyErr_SetString(PyExc_TypeError,
23837 359a57e96d5e Fix typo in exception message Neal Norwitz <nnorwitz@gmail.com> parents: 23832 diff changeset	5986 "cannot use unicode as modifiable buffer");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5987 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5988 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5989
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5990 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5991 unicode_buffer_getsegcount(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5992 int *lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5993 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5994 if (lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5995 *lenp = PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5996 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5997 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5998
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5999 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6000 unicode_buffer_getcharbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6001 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6002 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6003 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6004 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6005
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6006 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6007 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6008 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6009 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6010 }
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	6011 str = _PyUnicode_AsDefaultEncodedString((PyObject *)self, NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6012 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6013 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6014 ptr = (void ) PyString_AS_STRING(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6015 return PyString_GET_SIZE(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6016 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6017
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6018 /* Helpers for PyUnicode_Format() */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6019
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6020 static PyObject *
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	6021 getnextarg(PyObject args, int arglen, int p_argidx)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6022 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6023 int argidx = *p_argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6024 if (argidx < arglen) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6025 (*p_argidx)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6026 if (arglen < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6027 return args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6028 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6029 return PyTuple_GetItem(args, argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6030 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6031 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6032 "not enough arguments for format string");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6033 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6034 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6035
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6036 #define F_LJUST (1<<0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6037 #define F_SIGN (1<<1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6038 #define F_BLANK (1<<2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6039 #define F_ALT (1<<3)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6040 #define F_ZERO (1<<4)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6041
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6042 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6043 int usprintf(register Py_UNICODE buffer, char format, ...)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6044 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6045 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6046 int len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6047 va_list va;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6048 char *charbuffer;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6049 va_start(va, format);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6050
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6051 /* First, format the string as char array, then expand to Py_UNICODE
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6052 array. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6053 charbuffer = (char *)buffer;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6054 len = vsprintf(charbuffer, format, va);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6055 for (i = len - 1; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6056 buffer[i] = (Py_UNICODE) charbuffer[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6057
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6058 va_end(va);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6059 return len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6060 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6061
24736 ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6062 /* XXX To save some code duplication, formatfloat/long/int could have been
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6063 shared with stringobject.c, converting from 8-bit to Unicode after the
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6064 formatting is done. */
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6065
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6066 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6067 formatfloat(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6068 size_t buflen,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6069 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6070 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6071 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6072 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6073 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6074 /* fmt = '%#.' + `prec` + `type`
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6075 worst case length = 3 + 10 (len of INT_MAX) + 1 = 14 (use 20)*/
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6076 char fmt[20];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6077 double x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6078
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6079 x = PyFloat_AsDouble(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6080 if (x == -1.0 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6081 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6082 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6083 prec = 6;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6084 if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6085 type = 'g';
26825 e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6086 /* Worst case length calc to ensure no buffer overrun:
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6087
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6088 'g' formats:
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6089 fmt = %#.<prec>g
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6090 buf = '-' + [0-9]*prec + '.' + 'e+' + (longest exp
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6091 for any double rep.)
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6092 len = 1 + prec + 1 + 2 + 5 = 9 + prec
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6093
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6094 'f' formats:
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6095 buf = '-' + [0-9]x + '.' + [0-9]prec (with x < 50)
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6096 len = 1 + 50 + 1 + prec = 52 + prec
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6097
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6098 If prec=0 the effective precision is 1 (the leading digit is
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6099 always given), therefore increase the length by one.
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6100
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6101 */
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6102 if ((type == 'g' && buflen <= (size_t)10 + (size_t)prec) \|\|
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6103 (type == 'f' && buflen <= (size_t)53 + (size_t)prec)) {
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6104 PyErr_SetString(PyExc_OverflowError,
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6105 "formatted float is too long (precision too large?)");
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6106 return -1;
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6107 }
21082 c8d775b9ff92 formatfloat(), formatint(): Conversion of sprintf() to PyOS_snprintf() Barry Warsaw <barry@python.org> parents: 21067 diff changeset	6108 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%d%c",
26825 e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6109 (flags&F_ALT) ? "#" : "",
e153961f961f Patch for bug #659709: bogus computation of float length Marc-André Lemburg <mal@egenix.com> parents: 26274 diff changeset	6110 prec, type);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6111 return usprintf(buf, fmt, x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6112 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6113
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6114 static PyObject*
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6115 formatlong(PyObject *val, int flags, int prec, int type)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6116 {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6117 char *buf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6118 int i, len;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6119 PyObject str; / temporary string object. */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6120 PyUnicodeObject *result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6121
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6122 str = _PyString_FormatLong(val, flags, prec, type, &buf, &len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6123 if (!str)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6124 return NULL;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6125 result = _PyUnicode_New(len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6126 for (i = 0; i < len; i++)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6127 result->str[i] = buf[i];
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6128 result->str[len] = 0;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6129 Py_DECREF(str);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6130 return (PyObject*)result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6131 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6132
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6133 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6134 formatint(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6135 size_t buflen,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6136 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6137 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6138 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6139 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6140 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6141 /* fmt = '%#.' + `prec` + 'l' + `type`
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6142 * worst case length = 3 + 19 (worst len of INT_MAX on 64-bit machine)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6143 * + 1 + 1
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6144 * = 24
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6145 */
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6146 char fmt[64]; /* plenty big enough! */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6147 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6148
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6149 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6150 if (x == -1 && PyErr_Occurred())
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6151 return -1;
24736 ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6152 if (x < 0 && type != 'd' && type != 'i') {
24831 a602e4878b4e More changes of DeprecationWarning to FutureWarning. Guido van Rossum <guido@python.org> parents: 24737 diff changeset	6153 if (PyErr_Warn(PyExc_FutureWarning,
24736 ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6154 "%u/%o/%x/%X of negative int will return "
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6155 "a signed string in Python 2.4 and up") < 0)
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6156 return -1;
ff05eef3a6d1 Implement stage B0 of PEP 237: add warnings for operations that Guido van Rossum <guido@python.org> parents: 24703 diff changeset	6157 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6158 if (prec < 0)
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6159 prec = 1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6160
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6161 /* buf = '+'/'-'/'0'/'0x' + '[0-9]'*max(prec,len(x in octal))
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6162 * worst case buf = '0x' + [0-9]*prec, where prec >= 11
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6163 */
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6164 if (buflen <= 13 \|\| buflen <= (size_t)2 + (size_t)prec) {
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6165 PyErr_SetString(PyExc_OverflowError,
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6166 "formatted integer is too long (precision too large?)");
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6167 return -1;
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6168 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6169
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6170 if ((flags & F_ALT) &&
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6171 (type == 'x' \|\| type == 'X')) {
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6172 /* When converting under %#x or %#X, there are a number
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6173 * of issues that cause pain:
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6174 * - when 0 is being converted, the C standard leaves off
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6175 * the '0x' or '0X', which is inconsistent with other
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6176 * %#x/%#X conversions and inconsistent with Python's
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6177 * hex() function
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6178 * - there are platforms that violate the standard and
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6179 * convert 0 with the '0x' or '0X'
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6180 * (Metrowerks, Compaq Tru64)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6181 * - there are platforms that give '0x' when converting
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6182 * under %#X, but convert 0 in accordance with the
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6183 * standard (OS/2 EMX)
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6184 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6185 * We can achieve the desired consistency by inserting our
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6186 * own '0x' or '0X' prefix, and substituting %x/%X in place
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6187 * of %#x/%#X.
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6188 *
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6189 * Note that this is the same approach as used in
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6190 * formatint() in stringobject.c
22053 49197fd76362 OS/2 EMX port changes (Objects part of patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 21886 diff changeset	6191 */
22087 5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6192 PyOS_snprintf(fmt, sizeof(fmt), "0%c%%.%dl%c",
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6193 type, prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6194 }
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6195 else {
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6196 PyOS_snprintf(fmt, sizeof(fmt), "%%%s.%dl%c",
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6197 (flags&F_ALT) ? "#" : "",
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6198 prec, type);
5b24a6ae2dc4 %#x/%#X format conversion cleanup (see patch #450267): Andrew MacIntyre <andymac@bullseye.apana.org.au> parents: 22053 diff changeset	6199 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6200 return usprintf(buf, fmt, x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6201 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6202
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6203 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6204 formatchar(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6205 size_t buflen,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6206 PyObject *v)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6207 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6208 /* presume that the buffer is at least 2 characters long */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6209 if (PyUnicode_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6210 if (PyUnicode_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6211 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6212 buf[0] = PyUnicode_AS_UNICODE(v)[0];
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6213 }
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6214
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6215 else if (PyString_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6216 if (PyString_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6217 goto onError;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6218 buf[0] = (Py_UNICODE)PyString_AS_STRING(v)[0];
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6219 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6220
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6221 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6222 /* Integer input truncated to a character */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6223 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6224 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6225 if (x == -1 && PyErr_Occurred())
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6226 goto onError;
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6227 #ifdef Py_UNICODE_WIDE
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6228 if (x < 0 \|\| x > 0x10ffff) {
28494 32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises Walter Dörwald <walter@livinglogic.de> parents: 28242 diff changeset	6229 PyErr_SetString(PyExc_OverflowError,
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6230 "%c arg not in range(0x110000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6231 "(wide Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6232 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6233 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6234 #else
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6235 if (x < 0 \|\| x > 0xffff) {
28494 32bea41fef68 Change formatchar(), so that u"%c" % 0xffffffff now raises Walter Dörwald <walter@livinglogic.de> parents: 28242 diff changeset	6236 PyErr_SetString(PyExc_OverflowError,
24737 eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6237 "%c arg not in range(0x10000) "
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6238 "(narrow Python build)");
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6239 return -1;
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6240 }
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6241 #endif
eaa87632b1e8 Add C API PyUnicode_FromOrdinal() which exposes unichr() at C level. Marc-André Lemburg <mal@egenix.com> parents: 24736 diff changeset	6242 buf[0] = (Py_UNICODE) x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6243 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6244 buf[1] = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6245 return 1;
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6246
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6247 onError:
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6248 PyErr_SetString(PyExc_TypeError,
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6249 "%c requires int or char");
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6250 return -1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6251 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6252
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6253 /* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6254
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6255 FORMATBUFLEN is the length of the buffer in which the floats, ints, &
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6256 chars are formatted. XXX This is a magic number. Each formatting
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6257 routine does bounds checking to ensure no overflow, but a better
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6258 solution may be to malloc a buffer of appropriate size for each
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6259 format. For now, the current solution is sufficient.
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6260 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6261 #define FORMATBUFLEN (size_t)120
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6262
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6263 PyObject PyUnicode_Format(PyObject format,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6264 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6265 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6266 Py_UNICODE fmt, res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6267 int fmtcnt, rescnt, reslen, arglen, argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6268 int args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6269 PyUnicodeObject *result = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6270 PyObject *dict = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6271 PyObject *uformat;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6272
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6273 if (format == NULL \|\| args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6274 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6275 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6276 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6277 uformat = PyUnicode_FromObject(format);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6278 if (uformat == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6279 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6280 fmt = PyUnicode_AS_UNICODE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6281 fmtcnt = PyUnicode_GET_SIZE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6282
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6283 reslen = rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6284 result = _PyUnicode_New(reslen);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6285 if (result == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6286 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6287 res = PyUnicode_AS_UNICODE(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6288
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6289 if (PyTuple_Check(args)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6290 arglen = PyTuple_Size(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6291 argidx = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6292 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6293 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6294 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6295 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6296 }
26164 63a08639dae2 Fix SF # 635969, No error "not all arguments converted" Neal Norwitz <nnorwitz@gmail.com> parents: 25895 diff changeset	6297 if (args->ob_type->tp_as_mapping && !PyTuple_Check(args) &&
63a08639dae2 Fix SF # 635969, No error "not all arguments converted" Neal Norwitz <nnorwitz@gmail.com> parents: 25895 diff changeset	6298 !PyObject_TypeCheck(args, &PyBaseString_Type))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6299 dict = args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6300
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6301 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6302 if (*fmt != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6303 if (--rescnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6304 rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6305 reslen += rescnt;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6306 if (_PyUnicode_Resize(&result, reslen) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6307 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6308 res = PyUnicode_AS_UNICODE(result) + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6309 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6310 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6311 res++ = fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6312 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6313 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6314 /* Got a format specifier */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6315 int flags = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6316 int width = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6317 int prec = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6318 Py_UNICODE c = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6319 Py_UNICODE fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6320 PyObject *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6321 PyObject *temp = NULL;
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6322 Py_UNICODE *pbuf;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6323 Py_UNICODE sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6324 int len;
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6325 Py_UNICODE formatbuf[FORMATBUFLEN]; /* For format{float,int,char}() */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6326
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6327 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6328 if (*fmt == '(') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6329 Py_UNICODE *keystart;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6330 int keylen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6331 PyObject *key;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6332 int pcount = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6333
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6334 if (dict == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6335 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6336 "format requires a mapping");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6337 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6338 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6339 ++fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6340 --fmtcnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6341 keystart = fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6342 /* Skip over balanced parentheses */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6343 while (pcount > 0 && --fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6344 if (*fmt == ')')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6345 --pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6346 else if (*fmt == '(')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6347 ++pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6348 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6349 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6350 keylen = fmt - keystart - 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6351 if (fmtcnt < 0 \|\| pcount > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6352 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6353 "incomplete format key");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6354 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6355 }
21028 3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	6356 #if 0
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6357 /* keys are converted to strings using UTF-8 and
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6358 then looked up since Python uses strings to hold
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6359 variables names etc. in its namespaces and we
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6360 wouldn't want to break common idioms. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6361 key = PyUnicode_EncodeUTF8(keystart,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6362 keylen,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6363 NULL);
21028 3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	6364 #else
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	6365 key = PyUnicode_FromUnicode(keystart, keylen);
3dfbbf42bf16 Fix for bug #438164: %-formatting using Unicode objects. Marc-André Lemburg <mal@egenix.com> parents: 20563 diff changeset	6366 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6367 if (key == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6368 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6369 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6370 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6371 args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6372 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6373 args = PyObject_GetItem(dict, key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6374 Py_DECREF(key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6375 if (args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6376 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6377 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6378 args_owned = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6379 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6380 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6381 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6382 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6383 switch (c = *fmt++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6384 case '-': flags \|= F_LJUST; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6385 case '+': flags \|= F_SIGN; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6386 case ' ': flags \|= F_BLANK; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6387 case '#': flags \|= F_ALT; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6388 case '0': flags \|= F_ZERO; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6389 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6390 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6391 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6392 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6393 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6394 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6395 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6396 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6397 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6398 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6399 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6400 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6401 width = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6402 if (width < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6403 flags \|= F_LJUST;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6404 width = -width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6405 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6406 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6407 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6408 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6409 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6410 width = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6411 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6412 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6413 if (c < '0' \|\| c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6414 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6415 if ((width*10) / 10 != width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6416 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6417 "width too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6418 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6419 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6420 width = width*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6421 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6422 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6423 if (c == '.') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6424 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6425 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6426 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6427 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6428 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6429 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6430 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6431 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6432 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6433 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6434 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6435 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6436 prec = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6437 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6438 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6439 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6440 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6441 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6442 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6443 prec = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6444 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6445 c = Py_CHARMASK(*fmt++);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6446 if (c < '0' \|\| c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6447 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6448 if ((prec*10) / 10 != prec) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6449 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6450 "prec too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6451 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6452 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6453 prec = prec*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6454 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6455 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6456 } /* prec */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6457 if (fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6458 if (c == 'h' \|\| c == 'l' \|\| c == 'L') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6459 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6460 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6461 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6462 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6463 if (fmtcnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6464 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6465 "incomplete format");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6466 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6467 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6468 if (c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6469 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6470 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6471 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6472 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6473 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6474 fill = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6475 switch (c) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6476
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6477 case '%':
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6478 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6479 /* presume that buffer length is at least 1 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6480 pbuf[0] = '%';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6481 len = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6482 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6483
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6484 case 's':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6485 case 'r':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6486 if (PyUnicode_Check(v) && c == 's') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6487 temp = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6488 Py_INCREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6489 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6490 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6491 PyObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6492 if (c == 's')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6493 temp = PyObject_Str(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6494 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6495 temp = PyObject_Repr(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6496 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6497 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6498 if (!PyString_Check(temp)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6499 /* XXX Note: this should never happen, since
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6500 PyObject_Repr() and PyObject_Str() assure
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6501 this */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6502 Py_DECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6503 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6504 "%s argument has non-string str()");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6505 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6506 }
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6507 unicode = PyUnicode_Decode(PyString_AS_STRING(temp),
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6508 PyString_GET_SIZE(temp),
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6509 NULL,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6510 "strict");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6511 Py_DECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6512 temp = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6513 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6514 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6515 }
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6516 pbuf = PyUnicode_AS_UNICODE(temp);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6517 len = PyUnicode_GET_SIZE(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6518 if (prec >= 0 && len > prec)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6519 len = prec;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6520 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6521
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6522 case 'i':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6523 case 'd':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6524 case 'u':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6525 case 'o':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6526 case 'x':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6527 case 'X':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6528 if (c == 'i')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6529 c = 'd';
15455 d0146d128897 Fox for SF bug #123859: %[duxXo] long formats inconsistent. Tim Peters <tim.peters@gmail.com> parents: 15001 diff changeset	6530 if (PyLong_Check(v)) {
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6531 temp = formatlong(v, flags, prec, c);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6532 if (!temp)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6533 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6534 pbuf = PyUnicode_AS_UNICODE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6535 len = PyUnicode_GET_SIZE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6536 /* unbounded ints can always produce
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6537 a sign character! */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6538 sign = 1;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6539 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6540 else {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6541 pbuf = formatbuf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6542 len = formatint(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6543 flags, prec, c, v);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6544 if (len < 0)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6545 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6546 /* only d conversion is signed */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6547 sign = c == 'd';
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6548 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6549 if (flags & F_ZERO)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6550 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6551 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6552
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6553 case 'e':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6554 case 'E':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6555 case 'f':
30217 21d3b06df793 SF bug #795506: Wrong handling of string format code for float values. Raymond Hettinger <python@rcn.com> parents: 30165 diff changeset	6556 case 'F':
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6557 case 'g':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6558 case 'G':
30217 21d3b06df793 SF bug #795506: Wrong handling of string format code for float values. Raymond Hettinger <python@rcn.com> parents: 30165 diff changeset	6559 if (c == 'F')
21d3b06df793 SF bug #795506: Wrong handling of string format code for float values. Raymond Hettinger <python@rcn.com> parents: 30165 diff changeset	6560 c = 'f';
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6561 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6562 len = formatfloat(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6563 flags, prec, c, v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6564 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6565 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6566 sign = 1;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6567 if (flags & F_ZERO)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6568 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6569 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6570
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6571 case 'c':
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6572 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6573 len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6574 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6575 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6576 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6577
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6578 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6579 PyErr_Format(PyExc_ValueError,
15579 31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the Andrew M. Kuchling <amk@amk.ca> parents: 15455 diff changeset	6580 "unsupported format character '%c' (0x%x) "
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the Andrew M. Kuchling <amk@amk.ca> parents: 15455 diff changeset	6581 "at index %i",
25248 fc2b73802f7b Fix part of SF bug # 544248 gcc warning in unicodeobject.c Neal Norwitz <nnorwitz@gmail.com> parents: 25233 diff changeset	6582 (31<=c && c<=126) ? (char)c : '?',
25375 4df0e691f4da Add cast to avoid compiler warning. Marc-André Lemburg <mal@egenix.com> parents: 25248 diff changeset	6583 (int)c,
25233 8d751b62ecc4 Fix warnings on 64-bit platforms about casts from pointers to ints. Guido van Rossum <guido@python.org> parents: 25120 diff changeset	6584 (int)(fmt -1 - PyUnicode_AS_UNICODE(uformat)));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6585 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6586 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6587 if (sign) {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6588 if (pbuf == '-' \|\| pbuf == '+') {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	6589 sign = *pbuf++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6590 len--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6591 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6592 else if (flags & F_SIGN)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6593 sign = '+';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6594 else if (flags & F_BLANK)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6595 sign = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6596 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6597 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6598 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6599 if (width < len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6600 width = len;
25719 f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	6601 if (rescnt - (sign != 0) < width) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6602 reslen -= rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6603 rescnt = width + fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6604 reslen += rescnt;
25719 f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	6605 if (reslen < 0) {
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	6606 Py_DECREF(result);
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	6607 return PyErr_NoMemory();
f532e9b69551 Fix a nasty endcase reported by Armin Rigo in SF bug 618623: Guido van Rossum <guido@python.org> parents: 25375 diff changeset	6608 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6609 if (_PyUnicode_Resize(&result, reslen) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6610 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6611 res = PyUnicode_AS_UNICODE(result)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6612 + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6613 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6614 if (sign) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6615 if (fill != ' ')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6616 *res++ = sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6617 rescnt--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6618 if (width > len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6619 width--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6620 }
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6621 if ((flags & F_ALT) && (c == 'x' \|\| c == 'X')) {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6622 assert(pbuf[0] == '0');
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6623 assert(pbuf[1] == c);
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6624 if (fill != ' ') {
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6625 res++ = pbuf++;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6626 res++ = pbuf++;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6627 }
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6628 rescnt -= 2;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6629 width -= 2;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6630 if (width < 0)
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6631 width = 0;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6632 len -= 2;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6633 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6634 if (width > len && !(flags & F_LJUST)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6635 do {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6636 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6637 *res++ = fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6638 } while (--width > len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6639 }
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6640 if (fill == ' ') {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6641 if (sign)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6642 *res++ = sign;
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6643 if ((flags & F_ALT) && (c == 'x' \|\| c == 'X')) {
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6644 assert(pbuf[0] == '0');
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	6645 assert(pbuf[1] == c);
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6646 res++ = pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6647 res++ = pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6648 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	6649 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6650 Py_UNICODE_COPY(res, pbuf, len);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6651 res += len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6652 rescnt -= len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6653 while (--width >= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6654 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6655 *res++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6656 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6657 if (dict && (argidx < arglen) && c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6658 PyErr_SetString(PyExc_TypeError,
23494 76f376bea5e5 Patch 549187. Improve string formatting error message. Raymond Hettinger <python@rcn.com> parents: 23295 diff changeset	6659 "not all arguments converted during string formatting");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6660 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6661 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6662 Py_XDECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6663 } /* '%' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6664 } /* until end */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6665 if (argidx < arglen && !dict) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6666 PyErr_SetString(PyExc_TypeError,
23494 76f376bea5e5 Patch 549187. Improve string formatting error message. Raymond Hettinger <python@rcn.com> parents: 23295 diff changeset	6667 "not all arguments converted during string formatting");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6668 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6669 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6670
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6671 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6672 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6673 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6674 Py_DECREF(uformat);
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6675 if (_PyUnicode_Resize(&result, reslen - rescnt))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	6676 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6677 return (PyObject *)result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6678
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6679 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6680 Py_XDECREF(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6681 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6682 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6683 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6684 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6685 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6686 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6687
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6688 static PyBufferProcs unicode_as_buffer = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6689 (getreadbufferproc) unicode_buffer_getreadbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6690 (getwritebufferproc) unicode_buffer_getwritebuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6691 (getsegcountproc) unicode_buffer_getsegcount,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6692 (getcharbufferproc) unicode_buffer_getcharbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6693 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6694
24280 1598a1e473e2 staticforward bites the dust. Jeremy Hylton <jeremy@alum.mit.edu> parents: 24071 diff changeset	6695 static PyObject *
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6696 unicode_subtype_new(PyTypeObject type, PyObject args, PyObject *kwds);
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6697
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6698 static PyObject *
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6699 unicode_new(PyTypeObject type, PyObject args, PyObject *kwds)
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6700 {
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6701 PyObject *x = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6702 static char *kwlist[] = {"string", "encoding", "errors", 0};
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6703 char *encoding = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6704 char *errors = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6705
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6706 if (type != &PyUnicode_Type)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6707 return unicode_subtype_new(type, args, kwds);
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6708 if (!PyArg_ParseTupleAndKeywords(args, kwds, "\|Oss:unicode",
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6709 kwlist, &x, &encoding, &errors))
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6710 return NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6711 if (x == NULL)
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6712 return (PyObject *)_PyUnicode_New(0);
20554 0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	6713 if (encoding == NULL && errors == NULL)
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	6714 return PyObject_Unicode(x);
0336ee5b46d4 SF patch #470578: Fixes to synchronize unicode() and str() Guido van Rossum <guido@python.org> parents: 20277 diff changeset	6715 else
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6716 return PyUnicode_FromEncodedObject(x, encoding, errors);
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6717 }
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6718
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6719 static PyObject *
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6720 unicode_subtype_new(PyTypeObject type, PyObject args, PyObject *kwds)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6721 {
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6722 PyUnicodeObject tmp, pnew;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6723 int n;
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6724
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6725 assert(PyType_IsSubtype(type, &PyUnicode_Type));
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6726 tmp = (PyUnicodeObject *)unicode_new(&PyUnicode_Type, args, kwds);
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6727 if (tmp == NULL)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6728 return NULL;
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6729 assert(PyUnicode_Check(tmp));
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6730 pnew = (PyUnicodeObject *) type->tp_alloc(type, n = tmp->length);
29641 4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	6731 if (pnew == NULL) {
4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	6732 Py_DECREF(tmp);
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6733 return NULL;
29641 4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	6734 }
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6735 pnew->str = PyMem_NEW(Py_UNICODE, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6736 if (pnew->str == NULL) {
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6737 _Py_ForgetReference((PyObject *)pnew);
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	6738 PyObject_Del(pnew);
29641 4909aa64405f SF patch 703666: Several objects don't decref tmp on failure in subtype_new Raymond Hettinger <python@rcn.com> parents: 29164 diff changeset	6739 Py_DECREF(tmp);
27826 7dffcc498c45 Add more missing PyErr_NoMemory() after failled memory allocs Neal Norwitz <nnorwitz@gmail.com> parents: 27774 diff changeset	6740 return PyErr_NoMemory();
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6741 }
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6742 Py_UNICODE_COPY(pnew->str, tmp->str, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6743 pnew->length = n;
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6744 pnew->hash = tmp->hash;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6745 Py_DECREF(tmp);
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	6746 return (PyObject *)pnew;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6747 }
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	6748
23832 ed1ba37a46b7 Patch #568124: Add doc string macros. Martin v. Löwis <martin@v.loewis.de> parents: 23778 diff changeset	6749 PyDoc_STRVAR(unicode_doc,
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6750 "unicode(string [, encoding[, errors]]) -> object\n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6751 \n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6752 Create a new Unicode object from the given encoded string.\n\
24392 0a327b768868 tighten up the unicode object's docstring a tad Skip Montanaro <skip@pobox.com> parents: 24280 diff changeset	6753 encoding defaults to the current default string encoding.\n\
0a327b768868 tighten up the unicode object's docstring a tad Skip Montanaro <skip@pobox.com> parents: 24280 diff changeset	6754 errors can be 'strict', 'replace' or 'ignore' and defaults to 'strict'.");
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6755
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6756 PyTypeObject PyUnicode_Type = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6757 PyObject_HEAD_INIT(&PyType_Type)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6758 0, /* ob_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6759 "unicode", /* tp_name */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6760 sizeof(PyUnicodeObject), /* tp_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6761 0, /* tp_itemsize */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6762 /* Slots */
20277 2983b54256c7 Enable GC for new-style instances. This touches lots of files, since Guido van Rossum <guido@python.org> parents: 20018 diff changeset	6763 (destructor)unicode_dealloc, /* tp_dealloc */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6764 0, /* tp_print */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6765 0, /* tp_getattr */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6766 0, /* tp_setattr */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6767 (cmpfunc) unicode_compare, /* tp_compare */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6768 (reprfunc) unicode_repr, /* tp_repr */
26274 958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	6769 &unicode_as_number, /* tp_as_number */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6770 &unicode_as_sequence, /* tp_as_sequence */
23778 245224d1b8c9 This is my nearly two year old patch Michael W. Hudson <mwh@python.net> parents: 23562 diff changeset	6771 &unicode_as_mapping, /* tp_as_mapping */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6772 (hashfunc) unicode_hash, /* tp_hash*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6773 0, /* tp_call*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6774 (reprfunc) unicode_str, /* tp_str */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6775 PyObject_GenericGetAttr, /* tp_getattro */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6776 0, /* tp_setattro */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6777 &unicode_as_buffer, /* tp_as_buffer */
26274 958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	6778 Py_TPFLAGS_DEFAULT \| Py_TPFLAGS_CHECKTYPES \|
958d07085dcf Add nb_remainder (i.e. __mod__) slot to unicode type. Fixes SF bug #615506. Neil Schemenauer <nascheme@enme.ucalgary.ca> parents: 26164 diff changeset	6779 Py_TPFLAGS_BASETYPE, /* tp_flags */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6780 unicode_doc, /* tp_doc */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6781 0, /* tp_traverse */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6782 0, /* tp_clear */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6783 0, /* tp_richcompare */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6784 0, /* tp_weaklistoffset */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6785 0, /* tp_iter */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6786 0, /* tp_iternext */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6787 unicode_methods, /* tp_methods */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6788 0, /* tp_members */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6789 0, /* tp_getset */
23542 220b98f06415 - A new type object, 'string', is added. This is a common base type Guido van Rossum <guido@python.org> parents: 23494 diff changeset	6790 &PyBaseString_Type, /* tp_base */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6791 0, /* tp_dict */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6792 0, /* tp_descr_get */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6793 0, /* tp_descr_set */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6794 0, /* tp_dictoffset */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6795 0, /* tp_init */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6796 0, /* tp_alloc */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	6797 unicode_new, /* tp_new */
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	6798 PyObject_Del, /* tp_free */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6799 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6800
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6801 /* Initialize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6802
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	6803 void _PyUnicode_Init(void)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6804 {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6805 int i;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6806
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	6807 /* Init the implementation */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6808 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6809 unicode_freelist_size = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6810 unicode_empty = _PyUnicode_New(0);
12876 243593f11fcf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12646 diff changeset	6811 strcpy(unicode_default_encoding, "ascii");
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6812 for (i = 0; i < 256; i++)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6813 unicode_latin1[i] = NULL;
23542 220b98f06415 - A new type object, 'string', is added. This is a common base type Guido van Rossum <guido@python.org> parents: 23494 diff changeset	6814 if (PyType_Ready(&PyUnicode_Type) < 0)
220b98f06415 - A new type object, 'string', is added. This is a common base type Guido van Rossum <guido@python.org> parents: 23494 diff changeset	6815 Py_FatalError("Can't initialize 'unicode'");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6816 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6817
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6818 /* Finalize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6819
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6820 void
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	6821 _PyUnicode_Fini(void)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6822 {
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	6823 PyUnicodeObject *u;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6824 int i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6825
14998 c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free Guido van Rossum <guido@python.org> parents: 14869 diff changeset	6826 Py_XDECREF(unicode_empty);
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free Guido van Rossum <guido@python.org> parents: 14869 diff changeset	6827 unicode_empty = NULL;
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	6828
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6829 for (i = 0; i < 256; i++) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6830 if (unicode_latin1[i]) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6831 Py_DECREF(unicode_latin1[i]);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6832 unicode_latin1[i] = NULL;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6833 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6834 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	6835
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	6836 for (u = unicode_freelist; u != NULL;) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6837 PyUnicodeObject *v = u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6838 u = (PyUnicodeObject *)u;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	6839 if (v->str)
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	6840 PyMem_DEL(v->str);
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	6841 Py_XDECREF(v->defenc);
22981 dc1fb19d990d Remove PyMalloc_. Neil Schemenauer <nascheme@enme.ucalgary.ca>* parents: 22938 diff changeset	6842 PyObject_Del(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6843 }
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6844 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	6845 unicode_freelist_size = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6846 }
29164 4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	6847
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	6848 /*
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	6849 Local variables:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	6850 c-basic-offset: 4
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	6851 indent-tabs-mode: nil
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	6852 End:
4d9017ef7b5f Consider \U-escapes in raw-unicode-escape. Fixes #444514. Martin v. Löwis <martin@v.loewis.de> parents: 28575 diff changeset	6853 */

Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodeobject.c @ 30446:a8a1bef407eb 2.3