cpython-lru_cache-weakref: Objects/unicodeobject.c annotate

annotate Objects/unicodeobject.c @ 20010:d03c19db3094 legacy-trunk

Document new file() constructor, with the body of open()'s text, plus a "new in 2.2" blurb at the end. Replace open()'s text by pointing back to file().

author	Tim Peters <tim.peters@gmail.com>
date	Thu, 20 Sep 2001 19:55:29 +0000
parents	56ffee1eeeec
children	8aa1ada33f64

rev	line source
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1 /*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3 Unicode implementation based on original code by Fredrik Lundh,
12646 17d1a3f1f085 Minimal change so I can add the rest of MAL's checkin message: Fred Drake <fdrake@acm.org> parents: 12645 diff changeset	4 modified by Marc-Andre Lemburg <mal@lemburg.com> according to the
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5 Unicode Integration Proposal (see file Misc/unicode.txt).
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	6
13828 254f8b7c4a83 Changing the CNRI copyright notice according to CNRI's instructions. Guido van Rossum <guido@python.org> parents: 13761 diff changeset	7 Copyright (c) Corporation for National Research Initiatives.
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	8
16029 55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	9 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	10 The original string type implementation is:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	11
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	12 Copyright (c) 1999 by Secret Labs AB
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	13 Copyright (c) 1999 by Fredrik Lundh
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	14
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	15 By obtaining, using, and/or copying this software and/or its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	16 associated documentation, you agree that you have read, understood,
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	17 and will comply with the following terms and conditions:
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	18
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	19 Permission to use, copy, modify, and distribute this software and its
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	20 associated documentation for any purpose and without fee is hereby
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	21 granted, provided that the above copyright notice appears in all
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	22 copies, and that both that copyright notice and this permission notice
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	23 appear in supporting documentation, and that the name of Secret Labs
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	24 AB or the author not be used in advertising or publicity pertaining to
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	25 distribution of the software without specific, written prior
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	26 permission.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	27
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	28 SECRET LABS AB AND THE AUTHOR DISCLAIMS ALL WARRANTIES WITH REGARD TO
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	29 THIS SOFTWARE, INCLUDING ALL IMPLIED WARRANTIES OF MERCHANTABILITY AND
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	30 FITNESS. IN NO EVENT SHALL SECRET LABS AB OR THE AUTHOR BE LIABLE FOR
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	31 ANY SPECIAL, INDIRECT OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	32 WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	33 ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	34 OF OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE.
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	35 --------------------------------------------------------------------
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	36
55d89a8fe963 refactored the unicodeobject/ucnhash interface, to hide the Fredrik Lundh <fredrik@pythonware.com> parents: 15952 diff changeset	37 */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	38
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	39 #include "Python.h"
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	40
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	41 #include "unicodeobject.h"
13209 b11e0a898de0 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13196 diff changeset	42 #include "ucnhash.h"
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	43
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	44 #ifdef MS_WIN32
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	45 #include <windows.h>
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	46 #endif
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	47
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	48 /* Limit for the Unicode object free list */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	49
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	50 #define MAX_UNICODE_FREELIST_SIZE 1024
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	51
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	52 /* Limit for the Unicode object free list stay alive optimization.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	53
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	54 The implementation will keep allocated Unicode memory intact for
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	55 all objects on the free list having a size less than this
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	56 limit. This reduces malloc() overhead for small Unicode objects.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	57
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	58 At worst this will result in MAX_UNICODE_FREELIST_SIZE *
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	59 (sizeof(PyUnicodeObject) + KEEPALIVE_SIZE_LIMIT +
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	60 malloc()-overhead) bytes of unused garbage.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	61
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	62 Setting the limit to 0 effectively turns the feature off.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	63
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	64 Note: This is an experimental feature ! If you get core dumps when
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	65 using Unicode objects, turn this feature off.
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	66
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	67 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	68
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	69 #define KEEPALIVE_SIZE_LIMIT 9
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	70
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	71 /* Endianness switches; defaults to little endian */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	72
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	73 #ifdef WORDS_BIGENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	74 # define BYTEORDER_IS_BIG_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	75 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	76 # define BYTEORDER_IS_LITTLE_ENDIAN
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	77 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	78
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	79 /* --- Globals ------------------------------------------------------------
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	80
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	81 The globals are initialized by the _PyUnicode_Init() API and should
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	82 not be used before calling that API.
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	83
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	84 */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	85
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	86 /* Free list for Unicode objects */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	87 static PyUnicodeObject *unicode_freelist;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	88 static int unicode_freelist_size;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	89
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	90 /* The empty Unicode object is shared to improve performance. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	91 static PyUnicodeObject *unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	92
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	93 /* Single character Unicode strings in the Latin-1 range are being
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	94 shared as well. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	95 static PyUnicodeObject *unicode_latin1[256];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	96
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	97 /* Default encoding to use and assume when NULL is passed as encoding
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	98 parameter; it is initialized by _PyUnicode_Init().
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	99
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	100 Always use the PyUnicode_SetDefaultEncoding() and
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	101 PyUnicode_GetDefaultEncoding() APIs to access this global.
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	102
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	103 */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	104 static char unicode_default_encoding[100];
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	105
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	106 Py_UNICODE
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	107 PyUnicode_GetMax(void)
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	108 {
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	109 #ifdef Py_UNICODE_WIDE
18298 982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	110 return 0x10FFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	111 #else
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	112 /* This is actually an illegal character, so it should
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	113 not be passed to unichr. */
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	114 return 0xFFFF;
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	115 #endif
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	116 }
982163c4e9d2 Encode surrogates in UTF-8 even for a wide Py_UNICODE. Martin v. Löwis <martin@v.loewis.de> parents: 18295 diff changeset	117
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	118 /* --- Unicode Object ----------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	119
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	120 static
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	121 int unicode_resize(register PyUnicodeObject *unicode,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	122 int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	123 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	124 void *oldstr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	125
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	126 /* Shortcut if there's nothing much to do. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	127 if (unicode->length == length)
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	128 goto reset;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	129
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	130 /* Resizing shared object (unicode_empty or single character
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	131 objects) in-place is not allowed. Use PyUnicode_Resize()
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	132 instead ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	133 if (unicode == unicode_empty \|\|
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	134 (unicode->length == 1 &&
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	135 unicode->str[0] < 256 &&
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	136 unicode_latin1[unicode->str[0]] == unicode)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	137 PyErr_SetString(PyExc_SystemError,
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	138 "can't resize shared unicode objects");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	139 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	140 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	141
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	142 /* We allocate one more byte to make sure the string is
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	143 Ux0000 terminated -- XXX is this needed ? */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	144 oldstr = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	145 PyMem_RESIZE(unicode->str, Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	146 if (!unicode->str) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	147 unicode->str = oldstr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	148 PyErr_NoMemory();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	149 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	150 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	151 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	152 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	153
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	154 reset:
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	155 /* Reset the object caches */
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	156 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	157 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	158 unicode->defenc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	159 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	160 unicode->hash = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	161
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	162 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	163 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	164
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	165 /* We allocate one more byte to make sure the string is
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	166 Ux0000 terminated -- XXX is this needed ?
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	167
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	168 XXX This allocator could further be enhanced by assuring that the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	169 free list never reduces its size below 1.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	170
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	171 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	172
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	173 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	174 PyUnicodeObject *_PyUnicode_New(int length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	175 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	176 register PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	177
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	178 /* Optimization for empty strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	179 if (length == 0 && unicode_empty != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	180 Py_INCREF(unicode_empty);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	181 return unicode_empty;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	182 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	183
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	184 /* Unicode freelist & memory allocation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	185 if (unicode_freelist) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	186 unicode = unicode_freelist;
12939 88004f25dc25 Vladimir MARANGOZOV <Vladimir.Marangozov@inrialpes.fr>: Marc-André Lemburg <mal@egenix.com> parents: 12924 diff changeset	187 unicode_freelist = (PyUnicodeObject *)unicode;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	188 unicode_freelist_size--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	189 if (unicode->str) {
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	190 /* Keep-Alive optimization: we only upsize the buffer,
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	191 never downsize it. */
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	192 if ((unicode->length < length) &&
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	193 unicode_resize(unicode, length)) {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	194 PyMem_DEL(unicode->str);
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	195 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	196 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	197 }
18120 c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	198 else {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	199 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
18120 c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	200 }
c29f3135a20c Fix a mis-indentation in _PyUnicode_New() that caused me to stare at Guido van Rossum <guido@python.org> parents: 18051 diff changeset	201 PyObject_INIT(unicode, &PyUnicode_Type);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	202 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	203 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	204 unicode = PyObject_NEW(PyUnicodeObject, &PyUnicode_Type);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	205 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	206 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	207 unicode->str = PyMem_NEW(Py_UNICODE, length + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	208 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	209
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	210 if (!unicode->str) {
8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	211 PyErr_NoMemory();
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	212 goto onError;
12511 8747f1c8b5fb Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12369 diff changeset	213 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	214 unicode->str[length] = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	215 unicode->length = length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	216 unicode->hash = -1;
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	217 unicode->defenc = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	218 return unicode;
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	219
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	220 onError:
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	221 _Py_ForgetReference((PyObject *)unicode);
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	222 PyObject_DEL(unicode);
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	223 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	224 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	225
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	226 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	227 void _PyUnicode_Free(register PyUnicodeObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	228 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	229 if (unicode_freelist_size < MAX_UNICODE_FREELIST_SIZE) {
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	230 /* Keep-Alive optimization */
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	231 if (unicode->length >= KEEPALIVE_SIZE_LIMIT) {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	232 PyMem_DEL(unicode->str);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	233 unicode->str = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	234 unicode->length = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	235 }
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	236 if (unicode->defenc) {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	237 Py_DECREF(unicode->defenc);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	238 unicode->defenc = NULL;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	239 }
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	240 /* Add to free list */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	241 (PyUnicodeObject *)unicode = unicode_freelist;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	242 unicode_freelist = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	243 unicode_freelist_size++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	244 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	245 else {
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	246 PyMem_DEL(unicode->str);
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	247 Py_XDECREF(unicode->defenc);
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	248 PyObject_DEL(unicode);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	249 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	250 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	251
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	252 int PyUnicode_Resize(PyObject **unicode,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	253 int length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	254 {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	255 register PyUnicodeObject *v;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	256
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	257 /* Argument checks */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	258 if (unicode == NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	259 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	260 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	261 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	262 v = (PyUnicodeObject )unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	263 if (v == NULL \|\| !PyUnicode_Check(v) \|\| v->ob_refcnt != 1) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	264 PyErr_BadInternalCall();
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	265 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	266 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	267
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	268 /* Resizing unicode_empty and single character objects is not
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	269 possible since these are being shared. We simply return a fresh
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	270 copy with the same Unicode content. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	271 if (v->length != length &&
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	272 (v == unicode_empty \|\| v->length == 1)) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	273 PyUnicodeObject *w = _PyUnicode_New(length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	274 if (w == NULL)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	275 return -1;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	276 Py_UNICODE_COPY(w->str, v->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	277 length < v->length ? length : v->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	278 unicode = (PyObject )w;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	279 return 0;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	280 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	281
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	282 /* Note that we don't have to modify *unicode for unshared Unicode
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	283 objects, since we can modify them in-place. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	284 return unicode_resize(v, length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	285 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	286
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	287 /* Internal API for use in unicodeobject.c only ! */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	288 #define _PyUnicode_Resize(unicodevar, length) \
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	289 PyUnicode_Resize(((PyObject **)(unicodevar)), length)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	290
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	291 PyObject PyUnicode_FromUnicode(const Py_UNICODE u,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	292 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	293 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	294 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	295
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	296 /* If the Unicode data is known at construction time, we can apply
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	297 some optimizations which share commonly used objects. */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	298 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	299
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	300 /* Optimization for empty strings */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	301 if (size == 0 && unicode_empty != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	302 Py_INCREF(unicode_empty);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	303 return (PyObject *)unicode_empty;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	304 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	305
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	306 /* Single character Unicode objects in the Latin-1 range are
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	307 shared when using this constructor */
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	308 if (size == 1 && *u < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	309 unicode = unicode_latin1[*u];
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	310 if (!unicode) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	311 unicode = _PyUnicode_New(1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	312 if (!unicode)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	313 return NULL;
18051 e897550b4748 Fixes [ #430986 ] Buglet in PyUnicode_FromUnicode. Marc-André Lemburg <mal@egenix.com> parents: 17995 diff changeset	314 unicode->str[0] = *u;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	315 unicode_latin1[*u] = unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	316 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	317 Py_INCREF(unicode);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	318 return (PyObject *)unicode;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	319 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	320 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	321
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	322 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	323 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	324 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	325
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	326 /* Copy the Unicode data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	327 if (u != NULL)
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	328 Py_UNICODE_COPY(unicode->str, u, size);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	329
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	330 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	331 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	332
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	333 #ifdef HAVE_WCHAR_H
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	334
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	335 PyObject PyUnicode_FromWideChar(register const wchar_t w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	336 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	337 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	338 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	340 if (w == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	341 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	342 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	343 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	344
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	345 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	346 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	347 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	348
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	349 /* Copy the wchar_t data into the new object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	350 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	351 memcpy(unicode->str, w, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	352 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	353 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	354 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	355 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	356 u = PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	357 for (i = size; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	358 u++ = w++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	359 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	360 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	361
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	362 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	363 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	364
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	365 int PyUnicode_AsWideChar(PyUnicodeObject *unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	366 register wchar_t *w,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	367 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	368 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	369 if (unicode == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	370 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	371 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	372 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	373 if (size > PyUnicode_GET_SIZE(unicode))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	374 size = PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	375 #ifdef HAVE_USABLE_WCHAR_T
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	376 memcpy(w, unicode->str, size * sizeof(wchar_t));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	377 #else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	378 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	379 register Py_UNICODE *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	380 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	381 u = PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	382 for (i = size; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	383 w++ = u++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	384 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	385 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	386
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	387 return size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	388 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	389
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	390 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	391
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	392 PyObject PyUnicode_FromObject(register PyObject obj)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	393 {
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	394 return PyUnicode_FromEncodedObject(obj, NULL, "strict");
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	395 }
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	396
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	397 PyObject PyUnicode_FromEncodedObject(register PyObject obj,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	398 const char *encoding,
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	399 const char *errors)
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	400 {
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	401 const char *s = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	402 int len;
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	403 int owned = 0;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	404 PyObject *v;
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	405 int reclevel;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	406
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	407 if (obj == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	408 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	409 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	410 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	411
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	412 /* Coerce object */
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	413 for (reclevel = 0; reclevel < 2; reclevel++) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	414
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	415 if (PyUnicode_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	416 if (encoding) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	417 PyErr_SetString(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	418 "decoding Unicode is not supported");
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	419 goto onError;
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	420 }
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	421 if (PyUnicode_CheckExact(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	422 Py_INCREF(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	423 v = obj;
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	424 }
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	425 else {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	426 /* For a subclass of unicode, return a true unicode object
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	427 with the same string value. */
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	428 v = PyUnicode_FromUnicode(PyUnicode_AS_UNICODE(obj),
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	429 PyUnicode_GET_SIZE(obj));
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	430 }
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	431 goto done;
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	432 }
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	433 else if (PyString_Check(obj)) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	434 s = PyString_AS_STRING(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	435 len = PyString_GET_SIZE(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	436 break;
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	437 }
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	438 else {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	439 PyObject *w;
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	440
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	441 /* Try char buffer interface */
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	442 if (PyObject_AsCharBuffer(obj, &s, &len))
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	443 PyErr_Clear();
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	444 else
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	445 break;
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	446
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	447 /* Mimic the behaviour of str(object) if everything else
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	448 fails (see PyObject_Str()); this also covers instances
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	449 which implement __str__. */
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	450 if (obj->ob_type->tp_str == NULL)
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	451 w = PyObject_Repr(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	452 else
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	453 w = (*obj->ob_type->tp_str)(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	454 if (w == NULL)
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	455 goto onError;
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	456 if (owned) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	457 Py_DECREF(obj);
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	458 }
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	459 obj = w;
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	460 owned = 1;
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	461 }
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	462 }
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	463
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	464 if (s == NULL) {
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	465 PyErr_Format(PyExc_TypeError,
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	466 "coercing to Unicode: __str__ recursion limit exceeded "
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	467 "(last type: %.80s)",
1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	468 obj->ob_type->tp_name);
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	469 goto onError;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	470 }
19999 1f4e2627b627 Implement the changes proposed in patch #413333. unicode(obj) now Marc-André Lemburg <mal@egenix.com> parents: 19998 diff changeset	471
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	472 /* Convert to Unicode */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	473 if (len == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	474 Py_INCREF(unicode_empty);
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	475 v = (PyObject *)unicode_empty;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	476 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	477 else
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	478 v = PyUnicode_Decode(s, len, encoding, errors);
15952 1f92d63b20f0 This patch adds a new builtin unistr() which behaves like str() Marc-André Lemburg <mal@egenix.com> parents: 15910 diff changeset	479
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	480 done:
13633 e1469b7a2e68 gcc is being stupid with if/else constructs Greg Stein <gstein@lyra.org> parents: 13628 diff changeset	481 if (owned) {
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	482 Py_DECREF(obj);
13633 e1469b7a2e68 gcc is being stupid with if/else constructs Greg Stein <gstein@lyra.org> parents: 13628 diff changeset	483 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	484 return v;
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	485
8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	486 onError:
13633 e1469b7a2e68 gcc is being stupid with if/else constructs Greg Stein <gstein@lyra.org> parents: 13628 diff changeset	487 if (owned) {
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	488 Py_DECREF(obj);
13633 e1469b7a2e68 gcc is being stupid with if/else constructs Greg Stein <gstein@lyra.org> parents: 13628 diff changeset	489 }
13401 8f1a2deeb9b0 Added new API PyUnicode_FromEncodedObject() which supports decoding Marc-André Lemburg <mal@egenix.com> parents: 13396 diff changeset	490 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	491 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	492
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	493 PyObject PyUnicode_Decode(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	494 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	495 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	496 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	497 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	498 PyObject buffer = NULL, unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	499
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	500 if (encoding == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	501 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	502
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	503 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	504 if (strcmp(encoding, "utf-8") == 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	505 return PyUnicode_DecodeUTF8(s, size, errors);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	506 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	507 return PyUnicode_DecodeLatin1(s, size, errors);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	508 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	509 return PyUnicode_DecodeASCII(s, size, errors);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	510
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	511 /* Decode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	512 buffer = PyBuffer_FromMemory((void *)s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	513 if (buffer == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	514 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	515 unicode = PyCodec_Decode(buffer, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	516 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	517 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	518 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	519 PyErr_Format(PyExc_TypeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	520 "decoder did not return an unicode object (type=%.400s)",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	521 unicode->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	522 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	523 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	524 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	525 Py_DECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	526 return unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	527
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	528 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	529 Py_XDECREF(buffer);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	530 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	531 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	532
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	533 PyObject PyUnicode_Encode(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	534 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	535 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	536 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	537 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	538 PyObject v, unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	539
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	540 unicode = PyUnicode_FromUnicode(s, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	541 if (unicode == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	542 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	543 v = PyUnicode_AsEncodedString(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	544 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	545 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	546 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	547
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	548 PyObject PyUnicode_AsEncodedString(PyObject unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	549 const char *encoding,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	550 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	551 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	552 PyObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	553
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	554 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	555 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	556 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	557 }
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	558
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	559 if (encoding == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	560 encoding = PyUnicode_GetDefaultEncoding();
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	561
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	562 /* Shortcuts for common default encodings */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	563 if (errors == NULL) {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	564 if (strcmp(encoding, "utf-8") == 0)
17995 837825a779c1 fix bogus indentation Jeremy Hylton <jeremy@alum.mit.edu> parents: 17887 diff changeset	565 return PyUnicode_AsUTF8String(unicode);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	566 else if (strcmp(encoding, "latin-1") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	567 return PyUnicode_AsLatin1String(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	568 else if (strcmp(encoding, "ascii") == 0)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	569 return PyUnicode_AsASCIIString(unicode);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	570 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	571
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	572 /* Encode via the codec registry */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	573 v = PyCodec_Encode(unicode, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	574 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	575 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	576 /* XXX Should we really enforce this ? */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	577 if (!PyString_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	578 PyErr_Format(PyExc_TypeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	579 "encoder did not return a string object (type=%.400s)",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	580 v->ob_type->tp_name);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	581 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	582 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	583 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	584 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	585
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	586 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	587 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	588 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	589
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	590 PyObject _PyUnicode_AsDefaultEncodedString(PyObject unicode,
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	591 const char *errors)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	592 {
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	593 PyObject v = ((PyUnicodeObject )unicode)->defenc;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	594
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	595 if (v)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	596 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	597 v = PyUnicode_AsEncodedString(unicode, NULL, errors);
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	598 if (v && errors == NULL)
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	599 ((PyUnicodeObject *)unicode)->defenc = v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	600 return v;
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	601 }
2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	602
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	603 Py_UNICODE PyUnicode_AsUnicode(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	604 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	605 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	606 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	607 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	608 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	609 return PyUnicode_AS_UNICODE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	610
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	611 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	612 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	613 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	614
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	615 int PyUnicode_GetSize(PyObject *unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	616 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	617 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	618 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	619 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	620 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	621 return PyUnicode_GET_SIZE(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	622
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	623 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	624 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	625 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	626
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	627 const char *PyUnicode_GetDefaultEncoding(void)
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	628 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	629 return unicode_default_encoding;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	630 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	631
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	632 int PyUnicode_SetDefaultEncoding(const char *encoding)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	633 {
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	634 PyObject *v;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	635
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	636 /* Make sure the encoding is valid. As side effect, this also
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	637 loads the encoding into the codec registry cache. */
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	638 v = _PyCodec_Lookup(encoding);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	639 if (v == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	640 goto onError;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	641 Py_DECREF(v);
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	642 strncpy(unicode_default_encoding,
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	643 encoding,
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	644 sizeof(unicode_default_encoding));
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	645 return 0;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	646
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	647 onError:
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	648 return -1;
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	649 }
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	650
19998 8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	651 /* --- UTF-7 Codec -------------------------------------------------------- */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	652
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	653 /* see RFC2152 for details */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	654
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	655 static
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	656 char utf7_special[128] = {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	657 /* indicate whether a UTF-7 character is special i.e. cannot be directly
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	658 encoded:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	659 0 - not special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	660 1 - special
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	661 2 - whitespace (optional)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	662 3 - RFC2152 Set O (optional) */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	663 1, 1, 1, 1, 1, 1, 1, 1, 1, 2, 2, 1, 1, 2, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	664 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	665 2, 3, 3, 3, 3, 3, 3, 0, 0, 0, 3, 1, 0, 0, 0, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	666 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 3, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	667 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	668 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 1, 3, 3, 3,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	669 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	670 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 3, 3, 3, 1, 1,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	671
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	672 };
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	673
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	674 #define SPECIAL(c, encodeO, encodeWS) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	675 (((c)>127 \|\| utf7_special[(c)] == 1) \|\| \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	676 (encodeWS && (utf7_special[(c)] == 2)) \|\| \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	677 (encodeO && (utf7_special[(c)] == 3)))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	678
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	679 #define B64(n) ("ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/"[(n) & 0x3f])
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	680 #define B64CHAR(c) (isalnum(c) \|\| (c) == '+' \|\| (c) == '/')
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	681 #define UB64(c) ((c) == '+' ? 62 : (c) == '/' ? 63 : (c) >= 'a' ? \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	682 (c) - 71 : (c) >= 'A' ? (c) - 65 : (c) + 4)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	683
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	684 #define ENCODE(out, ch, bits) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	685 while (bits >= 6) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	686 *out++ = B64(ch >> (bits-6)); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	687 bits -= 6; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	688 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	689
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	690 #define DECODE(out, ch, bits, surrogate) \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	691 while (bits >= 16) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	692 Py_UNICODE outCh = (Py_UNICODE) ((ch >> (bits-16)) & 0xffff); \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	693 bits -= 16; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	694 if (surrogate) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	695 /* We have already generated an error for the high surrogate
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	696 so let's not bother seeing if the low surrogate is correct or not */\
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	697 surrogate = 0; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	698 } else if (0xDC00 <= outCh && outCh <= 0xDFFF) { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	699 /* This is a surrogate pair. Unfortunately we can't represent \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	700 it in a 16-bit character */ \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	701 surrogate = 1; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	702 errmsg = "code pairs are not supported"; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	703 goto utf7Error; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	704 } else { \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	705 *out++ = outCh; \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	706 } \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	707 } \
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	708
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	709 static
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	710 int utf7_decoding_error(Py_UNICODE **dest,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	711 const char *errors,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	712 const char *details)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	713 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	714 if ((errors == NULL) \|\|
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	715 (strcmp(errors,"strict") == 0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	716 PyErr_Format(PyExc_UnicodeError,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	717 "UTF-7 decoding error: %.400s",
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	718 details);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	719 return -1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	720 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	721 else if (strcmp(errors,"ignore") == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	722 return 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	723 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	724 else if (strcmp(errors,"replace") == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	725 if (dest != NULL) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	726 **dest = Py_UNICODE_REPLACEMENT_CHARACTER;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	727 (*dest)++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	728 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	729 return 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	730 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	731 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	732 PyErr_Format(PyExc_ValueError,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	733 "UTF-7 decoding error; unknown error handling code: %.400s",
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	734 errors);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	735 return -1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	736 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	737 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	738
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	739 PyObject PyUnicode_DecodeUTF7(const char s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	740 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	741 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	742 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	743 const char *e;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	744 PyUnicodeObject *unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	745 Py_UNICODE *p;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	746 const char *errmsg = "";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	747 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	748 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	749 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	750 int surrogate = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	751
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	752 unicode = _PyUnicode_New(size);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	753 if (!unicode)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	754 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	755 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	756 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	757
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	758 p = unicode->str;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	759 e = s + size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	760
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	761 while (s < e) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	762 Py_UNICODE ch = *s;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	763
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	764 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	765 if ((ch == '-') \|\| !B64CHAR(ch)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	766 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	767 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	768
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	769 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	770 if (bitsleft >= 6) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	771 /* The shift sequence has a partial character in it. If
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	772 bitsleft < 6 then we could just classify it as padding
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	773 but that is not the case here */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	774
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	775 errmsg = "partial character in shift sequence";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	776 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	777 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	778 /* According to RFC2152 the remaining bits should be zero. We
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	779 choose to signal an error/insert a replacement character
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	780 here so indicate the potential of a misencoded character. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	781
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	782 /* On x86, a << b == a << (b%32) so make sure that bitsleft != 0 */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	783 if (bitsleft && charsleft << (sizeof(charsleft) * 8 - bitsleft)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	784 errmsg = "non-zero padding bits in shift sequence";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	785 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	786 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	787
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	788 if (ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	789 if ((s < e) && (*(s) == '-')) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	790 *p++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	791 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	792 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	793 } else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	794 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	795 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	796 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	797 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	798 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	799 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	800 charsleft = (charsleft << 6) \| UB64(ch);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	801 bitsleft += 6;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	802 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	803 /* p, charsleft, bitsleft, surrogate = */ DECODE(p, charsleft, bitsleft, surrogate);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	804 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	805 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	806 else if ( ch == '+' ) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	807 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	808 if (s < e && *s == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	809 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	810 *p++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	811 } else
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	812 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	813 inShift = 1;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	814 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	815 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	816 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	817 else if (SPECIAL(ch,0,0)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	818 errmsg = "unexpected special character";
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	819 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	820 goto utf7Error;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	821 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	822 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	823 *p++ = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	824 s++;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	825 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	826 continue;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	827 utf7Error:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	828 if (utf7_decoding_error(&p, errors, errmsg))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	829 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	830 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	831
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	832 if (inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	833 if (utf7_decoding_error(&p, errors, "unterminated shift sequence"))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	834 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	835 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	836
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	837 if (_PyUnicode_Resize(&unicode, p - unicode->str))
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	838 goto onError;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	839
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	840 return (PyObject *)unicode;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	841
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	842 onError:
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	843 Py_DECREF(unicode);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	844 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	845 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	846
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	847
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	848 PyObject PyUnicode_EncodeUTF7(const Py_UNICODE s,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	849 int size,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	850 int encodeSetO,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	851 int encodeWhiteSpace,
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	852 const char *errors)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	853 {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	854 PyObject *v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	855 /* It might be possible to tighten this worst case */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	856 unsigned int cbAllocated = 5 * size;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	857 int inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	858 int i = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	859 unsigned int bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	860 unsigned long charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	861 char * out;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	862 char * start;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	863
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	864 if (size == 0)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	865 return PyString_FromStringAndSize(NULL, 0);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	866
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	867 v = PyString_FromStringAndSize(NULL, cbAllocated);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	868 if (v == NULL)
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	869 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	870
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	871 start = out = PyString_AS_STRING(v);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	872 for (;i < size; ++i) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	873 Py_UNICODE ch = s[i];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	874
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	875 if (!inShift) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	876 if (ch == '+') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	877 *out++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	878 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	879 } else if (SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	880 charsleft = ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	881 bitsleft = 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	882 *out++ = '+';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	883 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	884 inShift = bitsleft > 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	885 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	886 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	887 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	888 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	889 if (!SPECIAL(ch, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	890 *out++ = B64(charsleft << (6-bitsleft));
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	891 charsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	892 bitsleft = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	893 /* Characters not in the BASE64 set implicitly unshift the sequence
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	894 so no '-' is required, except if the character is itself a '-' */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	895 if (B64CHAR(ch) \|\| ch == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	896 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	897 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	898 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	899 *out++ = (char) ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	900 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	901 bitsleft += 16;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	902 charsleft = (charsleft << 16) \| ch;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	903 /* out, charsleft, bitsleft = */ ENCODE(out, charsleft, bitsleft);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	904
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	905 /* If the next character is special then we dont' need to terminate
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	906 the shift sequence. If the next character is not a BASE64 character
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	907 or '-' then the shift sequence will be terminated implicitly and we
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	908 don't have to insert a '-'. */
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	909
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	910 if (bitsleft == 0) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	911 if (i + 1 < size) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	912 Py_UNICODE ch2 = s[i+1];
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	913
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	914 if (SPECIAL(ch2, encodeSetO, encodeWhiteSpace)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	915
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	916 } else if (B64CHAR(ch2) \|\| ch2 == '-') {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	917 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	918 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	919 } else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	920 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	921 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	922
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	923 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	924 else {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	925 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	926 inShift = 0;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	927 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	928 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	929 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	930 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	931 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	932 if (bitsleft) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	933 *out++= B64(charsleft << (6-bitsleft) );
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	934 *out++ = '-';
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	935 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	936
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	937 if (_PyString_Resize(&v, out - start)) {
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	938 Py_DECREF(v);
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	939 return NULL;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	940 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	941 return v;
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	942 }
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	943
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	944 #undef SPECIAL
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	945 #undef B64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	946 #undef B64CHAR
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	947 #undef UB64
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	948 #undef ENCODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	949 #undef DECODE
8a0c97453df4 Patch #435971: UTF-7 codec by Brian Quinlan. Marc-André Lemburg <mal@egenix.com> parents: 19886 diff changeset	950
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	951 /* --- UTF-8 Codec -------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	952
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	953 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	954 char utf8_code_length[256] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	955 /* Map UTF-8 encoded prefix byte to sequence length. zero means
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	956 illegal prefix. see RFC 2279 for details */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	957 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	958 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	959 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	960 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	961 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	962 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	963 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	964 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	965 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	966 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	967 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	968 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	969 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	970 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2, 2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	971 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3, 3,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	972 4, 4, 4, 4, 4, 4, 4, 4, 5, 5, 5, 5, 6, 6, 0, 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	973 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	974
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	975 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	976 int utf8_decoding_error(const char **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	977 Py_UNICODE **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	978 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	979 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	980 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	981 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	982 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	983 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	984 "UTF-8 decoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	985 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	986 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	987 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	988 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	989 (*source)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	990 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	991 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	992 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	993 (*source)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	994 **dest = Py_UNICODE_REPLACEMENT_CHARACTER;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	995 (*dest)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	996 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	997 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	998 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	999 PyErr_Format(PyExc_ValueError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	1000 "UTF-8 decoding error; unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1001 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1002 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1003 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1004 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1005
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1006 PyObject PyUnicode_DecodeUTF8(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1007 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1008 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1009 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1010 int n;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1011 const char *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1012 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1013 Py_UNICODE *p;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1014 const char *errmsg = "";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1015
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1016 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1017 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1018 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1019 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1020 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1021 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1022 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1023
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1024 /* Unpack UTF-8 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1025 p = unicode->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1026 e = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1027
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1028 while (s < e) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1029 Py_UCS4 ch = (unsigned char)*s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1030
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1031 if (ch < 0x80) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1032 *p++ = (Py_UNICODE)ch;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1033 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1034 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1035 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1036
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1037 n = utf8_code_length[ch];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1038
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1039 if (s + n > e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1040 errmsg = "unexpected end of data";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1041 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1042 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1043
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1044 switch (n) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1045
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1046 case 0:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1047 errmsg = "unexpected code byte";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1048 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1049
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1050 case 1:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1051 errmsg = "internal error";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1052 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1053
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1054 case 2:
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1055 if ((s[1] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1056 errmsg = "invalid data";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1057 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1058 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1059 ch = ((s[0] & 0x1f) << 6) + (s[1] & 0x3f);
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1060 if (ch < 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1061 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1062 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1063 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1064 else
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1065 *p++ = (Py_UNICODE)ch;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1066 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1067
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1068 case 3:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1069 if ((s[1] & 0xc0) != 0x80 \|\|
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1070 (s[2] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1071 errmsg = "invalid data";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1072 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1073 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1074 ch = ((s[0] & 0x0f) << 12) + ((s[1] & 0x3f) << 6) + (s[2] & 0x3f);
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1075 if (ch < 0x800 \|\| (ch >= 0xd800 && ch < 0xe000)) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1076 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1077 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1078 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1079 else
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1080 *p++ = (Py_UNICODE)ch;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1081 break;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1082
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1083 case 4:
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1084 if ((s[1] & 0xc0) != 0x80 \|\|
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1085 (s[2] & 0xc0) != 0x80 \|\|
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1086 (s[3] & 0xc0) != 0x80) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1087 errmsg = "invalid data";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1088 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1089 }
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1090 ch = ((s[0] & 0x7) << 18) + ((s[1] & 0x3f) << 12) +
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1091 ((s[2] & 0x3f) << 6) + (s[3] & 0x3f);
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1092 /* validate and convert to UTF-16 */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1093 if ((ch < 0x10000) /* minimum value allowed for 4
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1094 byte encoding */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1095 \|\| (ch > 0x10ffff)) /* maximum value allowed for
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1096 UTF-16 */
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1097 {
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1098 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1099 goto utf8Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1100 }
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1101 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1102 *p++ = (Py_UNICODE)ch;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1103 #else
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1104 /* compute and append the two surrogates: */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1105
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1106 /* translate from 10000..10FFFF to 0..FFFF */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1107 ch -= 0x10000;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1108
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1109 /* high surrogate = top 10 bits added to D800 */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1110 *p++ = (Py_UNICODE)(0xD800 + (ch >> 10));
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1111
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1112 /* low surrogate = bottom 10 bits added to DC00 */
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	1113 *p++ = (Py_UNICODE)(0xDC00 + (ch & 0x03FF));
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1114 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1115 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1116
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1117 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1118 /* Other sizes are only needed for UCS-4 */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1119 errmsg = "unsupported Unicode code range";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1120 goto utf8Error;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1121 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1122 s += n;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1123 continue;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1124
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1125 utf8Error:
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1126 if (utf8_decoding_error(&s, &p, errors, errmsg))
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1127 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1128 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1129
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1130 /* Adjust length */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	1131 if (_PyUnicode_Resize(&unicode, p - unicode->str))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1132 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1133
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1134 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1135
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1136 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1137 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1138 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1139 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1140
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1141 /* Not used anymore, now that the encoder supports UTF-16
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1142 surrogates. */
13633 e1469b7a2e68 gcc is being stupid with if/else constructs Greg Stein <gstein@lyra.org> parents: 13628 diff changeset	1143 #if 0
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1144 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1145 int utf8_encoding_error(const Py_UNICODE **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1146 char **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1147 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1148 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1149 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1150 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1151 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1152 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	1153 "UTF-8 encoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1154 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1155 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1156 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1157 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1158 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1159 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1160 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1161 **dest = '?';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1162 (*dest)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1163 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1164 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1165 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1166 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1167 "UTF-8 encoding error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	1168 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1169 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1170 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1171 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1172 }
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1173 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1174
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1175 PyObject PyUnicode_EncodeUTF8(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1176 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1177 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1178 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1179 PyObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1180 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1181 char *q;
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1182 Py_UCS4 ch2;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1183 unsigned int cbAllocated = 3 * size;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1184 unsigned int cbWritten = 0;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1185 int i = 0;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1186
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1187 v = PyString_FromStringAndSize(NULL, cbAllocated);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1188 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1189 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1190 if (size == 0)
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	1191 return v;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1192
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1193 p = q = PyString_AS_STRING(v);
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1194 while (i < size) {
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1195 Py_UCS4 ch = s[i++];
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1196 if (ch < 0x80) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1197 *p++ = (char) ch;
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1198 cbWritten++;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1199 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1200 else if (ch < 0x0800) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1201 *p++ = 0xc0 \| (ch >> 6);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1202 *p++ = 0x80 \| (ch & 0x3f);
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1203 cbWritten += 2;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1204 }
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1205 else if (ch < 0x10000) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1206 /* Check for high surrogate */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1207 if (0xD800 <= ch && ch <= 0xDBFF) {
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1208 if (i != size) {
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1209 ch2 = s[i];
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1210 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1211
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1212 if (cbWritten >= (cbAllocated - 4)) {
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1213 /* Provide enough room for some more
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1214 surrogates */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1215 cbAllocated += 4*10;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1216 if (_PyString_Resize(&v, cbAllocated))
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1217 goto onError;
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1218 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1219
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1220 /* combine the two values */
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1221 ch = ((ch - 0xD800)<<10 \| (ch2-0xDC00))+0x10000;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1222
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1223 *p++ = (char)((ch >> 18) \| 0xf0);
13633 e1469b7a2e68 gcc is being stupid with if/else constructs Greg Stein <gstein@lyra.org> parents: 13628 diff changeset	1224 *p++ = (char)(0x80 \| ((ch >> 12) & 0x3f));
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1225 i++;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1226 cbWritten += 4;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1227 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1228 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1229 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1230 else {
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1231 *p++ = (char)(0xe0 \| (ch >> 12));
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1232 cbWritten += 3;
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1233 }
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1234 *p++ = (char)(0x80 \| ((ch >> 6) & 0x3f));
9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	1235 *p++ = (char)(0x80 \| (ch & 0x3f));
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1236 } else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1237 *p++ = 0xf0 \| (ch>>18);
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1238 *p++ = 0x80 \| ((ch>>12) & 0x3f);
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1239 *p++ = 0x80 \| ((ch>>6) & 0x3f);
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1240 *p++ = 0x80 \| (ch & 0x3f);
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1241 cbWritten += 4;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1242 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1243 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1244 *p = '\0';
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1245 if (_PyString_Resize(&v, p - q))
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1246 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1247 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1248
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1249 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1250 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1251 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1252 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1253
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1254 PyObject PyUnicode_AsUTF8String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1255 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1256 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1257 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1258 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1259 }
14059 d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1260 return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1261 PyUnicode_GET_SIZE(unicode),
d0348a26b093 PyUnicode_AsUTF8String(): Don't need to explicitly incref str since Barry Warsaw <barry@python.org> parents: 13966 diff changeset	1262 NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1264
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1265 /* --- UTF-16 Codec ------------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1266
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1267 static
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1268 int utf16_decoding_error(Py_UNICODE **dest,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1269 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1270 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1271 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1272 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1273 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1274 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	1275 "UTF-16 decoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1276 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1277 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1278 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1279 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1280 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1281 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1282 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1283 if (dest) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1284 **dest = Py_UNICODE_REPLACEMENT_CHARACTER;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1285 (*dest)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1286 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1287 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1288 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1289 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1290 PyErr_Format(PyExc_ValueError,
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	1291 "UTF-16 decoding error; "
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	1292 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1293 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1294 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1295 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1296 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1297
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1298 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1299 PyUnicode_DecodeUTF16(const char *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1300 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1301 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1302 int *byteorder)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1303 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1304 PyUnicodeObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1305 Py_UNICODE *p;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1306 const unsigned char q, e;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1307 int bo = 0; /* assume native ordering by default */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1308 const char *errmsg = "";
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1309 /* Offsets from q for retrieving byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1310 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1311 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1312 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1313 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1314 #endif
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1315
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1316 /* size should be an even number */
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1317 if (size & 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1318 if (utf16_decoding_error(NULL, errors, "truncated data"))
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1319 return NULL;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1320 --size; /* else ignore the oddball byte */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1321 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1322
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1323 /* Note: size will always be longer than the resulting Unicode
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1324 character count */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1325 unicode = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1326 if (!unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1327 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1328 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1329 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1330
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1331 /* Unpack UTF-16 encoded data */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1332 p = unicode->str;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1333 q = (unsigned char *)s;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1334 e = q + size;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1335
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1336 if (byteorder)
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1337 bo = *byteorder;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1338
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1339 /* Check for BOM marks (U+FEFF) in the input and adjust current
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1340 byte order setting accordingly. In native mode, the leading BOM
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1341 mark is skipped, in all other modes, it is copied to the output
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1342 stream as-is (giving a ZWNBSP character). */
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1343 if (bo == 0) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1344 const Py_UNICODE bom = (q[ihi] << 8) \| q[ilo];
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1345 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1346 if (bom == 0xFEFF) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1347 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1348 bo = -1;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1349 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1350 else if (bom == 0xFFFE) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1351 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1352 bo = 1;
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1353 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1354 #else
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1355 if (bom == 0xFEFF) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1356 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1357 bo = 1;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1358 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1359 else if (bom == 0xFFFE) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1360 q += 2;
17887 fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1361 bo = -1;
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1362 }
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1363 #endif
fd5f83188778 This patch changes the behaviour of the UTF-16 codec family. Only the Marc-André Lemburg <mal@egenix.com> parents: 17700 diff changeset	1364 }
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1365
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1366 if (bo == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1367 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1368 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1369 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1370 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1371 else if (bo == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1372 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1373 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1374 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1375 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1376
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1377 while (q < e) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1378 Py_UNICODE ch = (q[ihi] << 8) \| q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1379 q += 2;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1380
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1381 if (ch < 0xD800 \|\| ch > 0xDFFF) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1382 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1383 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1384 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1385
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1386 /* UTF-16 code pair: */
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1387 if (q >= e) {
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1388 errmsg = "unexpected end of data";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1389 goto utf16Error;
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1390 }
18295 20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness Martin v. Löwis <martin@v.loewis.de> parents: 18292 diff changeset	1391 if (0xD800 <= ch && ch <= 0xDBFF) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1392 Py_UNICODE ch2 = (q[ihi] << 8) \| q[ilo];
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1393 q += 2;
18295 20e3a6e10c24 When decoding UTF-16, don't assume that the buffer is in native endianness Martin v. Löwis <martin@v.loewis.de> parents: 18292 diff changeset	1394 if (0xDC00 <= ch2 && ch2 <= 0xDFFF) {
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1395 #ifndef Py_UNICODE_WIDE
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1396 *p++ = ch;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1397 *p++ = ch2;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1398 #else
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1399 *p++ = (((ch & 0x3FF)<<10) \| (ch2 & 0x3FF)) + 0x10000;
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1400 #endif
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1401 continue;
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1402 }
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1403 else {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1404 errmsg = "illegal UTF-16 surrogate";
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1405 goto utf16Error;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1406 }
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1407
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1408 }
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1409 errmsg = "illegal encoding";
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1410 /* Fall through to report the error */
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1411
dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1412 utf16Error:
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1413 if (utf16_decoding_error(&p, errors, errmsg))
13635 dfb7ecfffe3d Fixed problems with UTF error reporting macros and some formatting bugs. Marc-André Lemburg <mal@egenix.com> parents: 13633 diff changeset	1414 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1415 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1416
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1417 if (byteorder)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1418 *byteorder = bo;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1419
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1420 /* Adjust length */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	1421 if (_PyUnicode_Resize(&unicode, p - unicode->str))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1422 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1423
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1424 return (PyObject *)unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1425
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1426 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1427 Py_DECREF(unicode);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1428 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1429 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1430
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1431 PyObject *
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1432 PyUnicode_EncodeUTF16(const Py_UNICODE *s,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1433 int size,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1434 const char *errors,
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1435 int byteorder)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1436 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1437 PyObject *v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1438 unsigned char *p;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1439 int i, pairs;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1440 /* Offsets from p for storing byte pairs in the right order. */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1441 #ifdef BYTEORDER_IS_LITTLE_ENDIAN
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1442 int ihi = 1, ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1443 #else
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1444 int ihi = 0, ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1445 #endif
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1446
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1447 #define STORECHAR(CH) \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1448 do { \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1449 p[ihi] = ((CH) >> 8) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1450 p[ilo] = (CH) & 0xff; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1451 p += 2; \
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1452 } while(0)
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1453
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1454 for (i = pairs = 0; i < size; i++)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1455 if (s[i] >= 0x10000)
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1456 pairs++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1457 v = PyString_FromStringAndSize(NULL,
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1458 2 * (size + pairs + (byteorder == 0)));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1459 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1460 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1461
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1462 p = (unsigned char *)PyString_AS_STRING(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1463 if (byteorder == 0)
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1464 STORECHAR(0xFEFF);
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1465 if (size == 0)
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	1466 return v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1467
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1468 if (byteorder == -1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1469 /* force LE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1470 ihi = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1471 ilo = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1472 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1473 else if (byteorder == 1) {
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1474 /* force BE */
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1475 ihi = 0;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1476 ilo = 1;
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1477 }
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1478
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1479 while (size-- > 0) {
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1480 Py_UNICODE ch = *s++;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1481 Py_UNICODE ch2 = 0;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1482 if (ch >= 0x10000) {
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1483 ch2 = 0xDC00 \| ((ch-0x10000) & 0x3FF);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1484 ch = 0xD800 \| ((ch-0x10000) >> 10);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1485 }
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1486 STORECHAR(ch);
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1487 if (ch2)
77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1488 STORECHAR(ch2);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1489 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1490 return v;
19121 77e405f3a942 SF patch #438013 Remove 2-byte Py_UCS2 assumptions Tim Peters <tim.peters@gmail.com> parents: 18933 diff changeset	1491 #undef STORECHAR
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1492 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1493
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1494 PyObject PyUnicode_AsUTF16String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1495 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1496 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1497 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1498 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1499 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1500 return PyUnicode_EncodeUTF16(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1501 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1502 NULL,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1503 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1504 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1505
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1506 /* --- Unicode Escape Codec ----------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1507
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1508 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1509 int unicodeescape_decoding_error(const char **source,
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1510 Py_UNICODE *x,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1511 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1512 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1513 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1514 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1515 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1516 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	1517 "Unicode-Escape decoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1518 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1519 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1520 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1521 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1522 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1523 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1524 else if (strcmp(errors,"replace") == 0) {
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1525 *x = Py_UNICODE_REPLACEMENT_CHARACTER;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1526 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1527 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1528 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1529 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1530 "Unicode-Escape decoding error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	1531 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1532 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1533 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1534 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1535 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1536
16200 3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1537 static _PyUnicode_Name_CAPI *ucnhash_CAPI = NULL;
13063 eb243443b884 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13049 diff changeset	1538
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1539 PyObject PyUnicode_DecodeUnicodeEscape(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1540 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1541 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1542 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1543 PyUnicodeObject *v;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1544 Py_UNICODE p, buf;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1545 const char *end;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1546 char* message;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1547 Py_UCS4 chr = 0xffffffff; /* in case 'getcode' messes up */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1548
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1549 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1550 Unicode string, so we start with size here and then reduce the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1551 length after conversion to the true value. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1552 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1553 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1554 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1555 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1556 return (PyObject *)v;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1557
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1558 p = buf = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1559 end = s + size;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1560
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1561 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1562 unsigned char c;
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1563 Py_UNICODE x;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1564 int i, digits;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1565
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1566 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1567 if (*s != '\\') {
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1568 p++ = (unsigned char) s++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1569 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1570 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1571
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1572 /* \ - Escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1573 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1574 switch (*s++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1575
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1576 /* \x escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1577 case '\n': break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1578 case '\\': *p++ = '\\'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1579 case '\'': *p++ = '\''; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1580 case '\"': *p++ = '\"'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1581 case 'b': *p++ = '\b'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1582 case 'f': p++ = '\014'; break; / FF */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1583 case 't': *p++ = '\t'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1584 case 'n': *p++ = '\n'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1585 case 'r': *p++ = '\r'; break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1586 case 'v': p++ = '\013'; break; / VT */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1587 case 'a': p++ = '\007'; break; / BEL, not classic C */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1588
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1589 /* \OOO (octal) escapes */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1590 case '0': case '1': case '2': case '3':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1591 case '4': case '5': case '6': case '7':
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1592 x = s[-1] - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1593 if ('0' <= s && s <= '7') {
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1594 x = (x<<3) + *s++ - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1595 if ('0' <= s && s <= '7')
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1596 x = (x<<3) + *s++ - '0';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1597 }
12544 599bfa4ffd49 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12511 diff changeset	1598 *p++ = x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1599 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1600
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1601 /* hex escapes */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1602 /* \xXX */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1603 case 'x':
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1604 digits = 2;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1605 message = "truncated \\xXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1606 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1607
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1608 /* \uXXXX */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1609 case 'u':
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1610 digits = 4;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1611 message = "truncated \\uXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1612 goto hexescape;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1613
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1614 /* \UXXXXXXXX */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1615 case 'U':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1616 digits = 8;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1617 message = "truncated \\UXXXXXXXX escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1618 hexescape:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1619 chr = 0;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1620 for (i = 0; i < digits; i++) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1621 c = (unsigned char) s[i];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1622 if (!isxdigit(c)) {
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1623 if (unicodeescape_decoding_error(&s, &x, errors, message))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1624 goto onError;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1625 chr = x;
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1626 i++;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1627 break;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1628 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1629 chr = (chr<<4) & ~0xF;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1630 if (c >= '0' && c <= '9')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1631 chr += c - '0';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1632 else if (c >= 'a' && c <= 'f')
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1633 chr += 10 + c - 'a';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1634 else
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1635 chr += 10 + c - 'A';
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1636 }
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1637 s += i;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1638 store:
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1639 /* when we get here, chr is a 32-bit unicode character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1640 if (chr <= 0xffff)
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1641 /* UCS-2 character */
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1642 *p++ = (Py_UNICODE) chr;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1643 else if (chr <= 0x10ffff) {
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1644 /* UCS-4 character. Either store directly, or as
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1645 surrogate pair. */
18324 3ba7a358ce02 use Py_UNICODE_WIDE instead of USE_UCS4_STORAGE and Py_UNICODE_SIZE Fredrik Lundh <fredrik@pythonware.com> parents: 18298 diff changeset	1646 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1647 *p++ = chr;
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1648 #else
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1649 chr -= 0x10000L;
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1650 *p++ = 0xD800 + (Py_UNICODE) (chr >> 10);
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	1651 *p++ = 0xDC00 + (Py_UNICODE) (chr & 0x03FF);
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1652 #endif
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1653 } else {
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1654 if (unicodeescape_decoding_error(
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1655 &s, &x, errors,
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1656 "illegal Unicode character")
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1657 )
ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1658 goto onError;
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1659 p++ = x; / store replacement character */
14388 ef2714bb0730 changed \x to consume exactly two hex digits, also for unicode Fredrik Lundh <fredrik@pythonware.com> parents: 14067 diff changeset	1660 }
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1661 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1662
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1663 /* \N{name} */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1664 case 'N':
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1665 message = "malformed \\N character escape";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1666 if (ucnhash_CAPI == NULL) {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1667 /* load the unicode data module */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1668 PyObject m, v;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1669 m = PyImport_ImportModule("unicodedata");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1670 if (m == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1671 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1672 v = PyObject_GetAttrString(m, "ucnhash_CAPI");
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1673 Py_DECREF(m);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1674 if (v == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1675 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1676 ucnhash_CAPI = PyCObject_AsVoidPtr(v);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1677 Py_DECREF(v);
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1678 if (ucnhash_CAPI == NULL)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1679 goto ucnhashError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1680 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1681 if (*s == '{') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1682 const char *start = s+1;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1683 /* look for the closing brace */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1684 while (*s != '}' && s < end)
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1685 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1686 if (s > start && s < end && *s == '}') {
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1687 /* found a name. look it up in the unicode database */
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1688 message = "unknown Unicode character name";
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1689 s++;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1690 if (ucnhash_CAPI->getcode(start, s-start-1, &chr))
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1691 goto store;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1692 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1693 }
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1694 if (unicodeescape_decoding_error(&s, &x, errors, message))
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1695 goto onError;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1696 *p++ = x;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1697 break;
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1698
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1699 default:
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1700 *p++ = '\\';
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1701 *p++ = (unsigned char)s[-1];
725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1702 break;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1703 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1704 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	1705 if (_PyUnicode_Resize(&v, (int)(p - buf)))
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	1706 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1707 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1708
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1709 ucnhashError:
16200 3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1710 PyErr_SetString(
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1711 PyExc_UnicodeError,
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1712 "\\N escapes not supported (can't load unicodedata module)"
3484670c37f2 Move uchhash functionality into unicodedata (after the recent Fredrik Lundh <fredrik@pythonware.com> parents: 16064 diff changeset	1713 );
16064 a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute Fredrik Lundh <fredrik@pythonware.com> parents: 16029 diff changeset	1714 return NULL;
a97aa9238771 Better error message if ucnhash cannot be found (obscure attribute Fredrik Lundh <fredrik@pythonware.com> parents: 16029 diff changeset	1715
16667 725238293869 reorganized PyUnicode_DecodeUnicodeEscape a bit (in order to make it Fredrik Lundh <fredrik@pythonware.com> parents: 16332 diff changeset	1716 onError:
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1717 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1718 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1719 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1720
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1721 /* Return a Unicode-Escape string version of the Unicode object.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1722
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1723 If quotes is true, the string is enclosed in u"" or u'' quotes as
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1724 appropriate.
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1725
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1726 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1727
12025 7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1728 static const Py_UNICODE findchar(const Py_UNICODE s,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1729 int size,
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1730 Py_UNICODE ch);
7994d9db7f4d On 17-Mar-2000, Marc-Andre Lemburg said: Barry Warsaw <barry@python.org> parents: 11997 diff changeset	1731
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1732 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1733 PyObject unicodeescape_string(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1734 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1735 int quotes)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1736 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1737 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1738 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1739
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1740 static const char *hexdigit = "0123456789abcdef";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1741
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1742 repr = PyString_FromStringAndSize(NULL, 2 + 6*size + 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1743 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1744 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1745
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1746 p = PyString_AS_STRING(repr);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1747
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1748 if (quotes) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1749 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1750 *p++ = (findchar(s, size, '\'') &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1751 !findchar(s, size, '"')) ? '"' : '\'';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1752 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1753 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1754 Py_UNICODE ch = *s++;
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1755
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1756 /* Escape quotes */
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1757 if (quotes &&
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1758 (ch == (Py_UNICODE) PyString_AS_STRING(repr)[1] \|\| ch == '\\')) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1759 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1760 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1761 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1762
18773 743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This Guido van Rossum <guido@python.org> parents: 18324 diff changeset	1763 #ifdef Py_UNICODE_WIDE
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1764 /* Map 21-bit characters to '\U00xxxxxx' */
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1765 else if (ch >= 0x10000) {
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1766 int offset = p - PyString_AS_STRING(repr);
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1767
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1768 /* Resize the string if necessary */
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1769 if (offset + 12 > PyString_GET_SIZE(repr)) {
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1770 if (_PyString_Resize(&repr, PyString_GET_SIZE(repr) + 100))
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1771 goto onError;
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1772 p = PyString_AS_STRING(repr) + offset;
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1773 }
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1774
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1775 *p++ = '\\';
ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1776 *p++ = 'U';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1777 *p++ = hexdigit[(ch >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1778 *p++ = hexdigit[(ch >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1779 *p++ = hexdigit[(ch >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1780 *p++ = hexdigit[(ch >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1781 *p++ = hexdigit[(ch >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1782 *p++ = hexdigit[(ch >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1783 *p++ = hexdigit[(ch >> 4) & 0x0000000F];
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1784 *p++ = hexdigit[ch & 0x0000000F];
081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1785 continue;
18292 ffbb9c918ac0 Support using UCS-4 as the Py_UNICODE type: Martin v. Löwis <martin@v.loewis.de> parents: 18280 diff changeset	1786 }
18773 743f8e1d953c #ifdef out generation of \U escapes unless Py_UNICODE_WIDE. This Guido van Rossum <guido@python.org> parents: 18324 diff changeset	1787 #endif
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1788 /* Map UTF-16 surrogate pairs to Unicode \UXXXXXXXX escapes */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1789 else if (ch >= 0xD800 && ch < 0xDC00) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1790 Py_UNICODE ch2;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1791 Py_UCS4 ucs;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1792
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1793 ch2 = *s++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1794 size--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1795 if (ch2 >= 0xDC00 && ch2 <= 0xDFFF) {
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1796 ucs = (((ch & 0x03FF) << 10) \| (ch2 & 0x03FF)) + 0x00010000;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1797 *p++ = '\\';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1798 *p++ = 'U';
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1799 *p++ = hexdigit[(ucs >> 28) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1800 *p++ = hexdigit[(ucs >> 24) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1801 *p++ = hexdigit[(ucs >> 20) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1802 *p++ = hexdigit[(ucs >> 16) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1803 *p++ = hexdigit[(ucs >> 12) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1804 *p++ = hexdigit[(ucs >> 8) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1805 *p++ = hexdigit[(ucs >> 4) & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1806 *p++ = hexdigit[ucs & 0x0000000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1807 continue;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1808 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1809 /* Fall through: isolated surrogates are copied as-is */
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1810 s--;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1811 size++;
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1812 }
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1813
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1814 /* Map 16-bit characters to '\uxxxx' */
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1815 if (ch >= 256) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1816 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1817 *p++ = 'u';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1818 *p++ = hexdigit[(ch >> 12) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1819 *p++ = hexdigit[(ch >> 8) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1820 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1821 *p++ = hexdigit[ch & 0x000F];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1822 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1823
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1824 /* Map special whitespace to '\t', \n', '\r' */
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1825 else if (ch == '\t') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1826 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1827 *p++ = 't';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1828 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1829 else if (ch == '\n') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1830 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1831 *p++ = 'n';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1832 }
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1833 else if (ch == '\r') {
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1834 *p++ = '\\';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1835 *p++ = 'r';
c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1836 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1837
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1838 /* Map non-printable US ASCII to '\xhh' */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1839 else if (ch < ' ' \|\| ch >= 128) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1840 *p++ = '\\';
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1841 *p++ = 'x';
18774 57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1842 *p++ = hexdigit[(ch >> 4) & 0x000F];
57684d3cefce Make the unicode-escape and the UTF-16 codecs handle surrogates Marc-André Lemburg <mal@egenix.com> parents: 18773 diff changeset	1843 *p++ = hexdigit[ch & 0x000F];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1844 }
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1845
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1846 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1847 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1848 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1849 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1850 if (quotes)
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1851 *p++ = PyString_AS_STRING(repr)[1];
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1852
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1853 *p = '\0';
18828 081d96fbcf43 Fix for bug #444493: u'\U00010001' segfaults with current CVS on Marc-André Lemburg <mal@egenix.com> parents: 18774 diff changeset	1854 if (_PyString_Resize(&repr, p - PyString_AS_STRING(repr)))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1855 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1856
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1857 return repr;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1858
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1859 onError:
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1860 Py_DECREF(repr);
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1861 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1862 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1863
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1864 PyObject PyUnicode_EncodeUnicodeEscape(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1865 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1866 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1867 return unicodeescape_string(s, size, 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1868 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1869
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1870 PyObject PyUnicode_AsUnicodeEscapeString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1871 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1872 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1873 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1874 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1875 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1876 return PyUnicode_EncodeUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1877 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1878 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1879
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1880 /* --- Raw Unicode Escape Codec ------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1881
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1882 PyObject PyUnicode_DecodeRawUnicodeEscape(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1883 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1884 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1885 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1886 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1887 Py_UNICODE p, buf;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1888 const char *end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1889 const char *bs;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1890
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1891 /* Escaped strings will always be longer than the resulting
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1892 Unicode string, so we start with size here and then reduce the
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1893 length after conversion to the true value. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1894 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1895 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1896 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1897 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1898 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1899 p = buf = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1900 end = s + size;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1901 while (s < end) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1902 unsigned char c;
13396 759f698740cc Fix to bug #393 (UTF16 codec didn't like empty strings) and Marc-André Lemburg <mal@egenix.com> parents: 13394 diff changeset	1903 Py_UNICODE x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1904 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1905
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1906 /* Non-escape characters are interpreted as Unicode ordinals */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1907 if (*s != '\\') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1908 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1909 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1910 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1911
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1912 /* \u-escapes are only interpreted iff the number of leading
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1913 backslashes if odd */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1914 bs = s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1915 for (;s < end;) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1916 if (*s != '\\')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1917 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1918 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1919 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1920 if (((s - bs) & 1) == 0 \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1921 s >= end \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1922 *s != 'u') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1923 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1924 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1925 p--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1926 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1927
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1928 /* \uXXXX with 4 hex digits */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1929 for (x = 0, i = 0; i < 4; i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1930 c = (unsigned char)s[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1931 if (!isxdigit(c)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1932 if (unicodeescape_decoding_error(&s, &x, errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1933 "truncated \\uXXXX"))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1934 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1935 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1936 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1937 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1938 x = (x<<4) & ~0xF;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1939 if (c >= '0' && c <= '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1940 x += c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1941 else if (c >= 'a' && c <= 'f')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1942 x += 10 + c - 'a';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1943 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1944 x += 10 + c - 'A';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1945 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1946 s += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1947 *p++ = x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1948 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	1949 if (_PyUnicode_Resize(&v, (int)(p - buf)))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1950 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1951 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1952
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1953 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1954 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1955 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1956 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1957
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1958 PyObject PyUnicode_EncodeRawUnicodeEscape(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1959 int size)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1960 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1961 PyObject *repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1962 char *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1963 char *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1964
16226 c7c404b35f33 Show '\011', '\012', and '\015' as '\t', '\n', '\r' in strings. Ka-Ping Yee <ping@zesty.ca> parents: 16200 diff changeset	1965 static const char *hexdigit = "0123456789abcdef";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1966
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1967 repr = PyString_FromStringAndSize(NULL, 6 * size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1968 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1969 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	1970 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	1971 return repr;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1972
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1973 p = q = PyString_AS_STRING(repr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1974 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1975 Py_UNICODE ch = *s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1976 /* Map 16-bit characters to '\uxxxx' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1977 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1978 *p++ = '\\';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1979 *p++ = 'u';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1980 *p++ = hexdigit[(ch >> 12) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1981 *p++ = hexdigit[(ch >> 8) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1982 *p++ = hexdigit[(ch >> 4) & 0xf];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1983 *p++ = hexdigit[ch & 15];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1984 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1985 /* Copy everything else as-is */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1986 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1987 *p++ = (char) ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1988 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1989 *p = '\0';
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1990 if (_PyString_Resize(&repr, p - q))
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1991 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1992
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1993 return repr;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1994
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1995 onError:
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1996 Py_DECREF(repr);
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	1997 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1998 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	1999
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2000 PyObject PyUnicode_AsRawUnicodeEscapeString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2001 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2002 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2003 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2004 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2005 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2006 return PyUnicode_EncodeRawUnicodeEscape(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2007 PyUnicode_GET_SIZE(unicode));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2008 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2009
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2010 /* --- Latin-1 Codec ------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2011
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2012 PyObject PyUnicode_DecodeLatin1(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2013 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2014 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2015 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2016 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2017 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2018
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2019 /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2020 if (size == 1 && (unsigned char)s < 256) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2021 Py_UNICODE r = (unsigned char)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2022 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2023 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2024
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2025 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2026 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2027 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2028 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2029 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2030 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2031 while (size-- > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2032 p++ = (unsigned char)s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2033 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2034
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2035 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2036 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2037 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2038 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2039
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2040 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2041 int latin1_encoding_error(const Py_UNICODE **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2042 char **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2043 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2044 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2045 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2046 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2047 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2048 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2049 "Latin-1 encoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2050 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2051 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2052 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2053 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2054 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2055 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2056 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2057 **dest = '?';
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2058 (*dest)++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2059 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2060 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2061 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2062 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2063 "Latin-1 encoding error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2064 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2065 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2066 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2067 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2068 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2069
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2070 PyObject PyUnicode_EncodeLatin1(const Py_UNICODE p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2071 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2072 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2073 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2074 PyObject *repr;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2075 char s, start;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2076
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2077 repr = PyString_FromStringAndSize(NULL, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2078 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2079 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2080 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2081 return repr;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2082
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2083 s = PyString_AS_STRING(repr);
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2084 start = s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2085 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2086 Py_UNICODE ch = *p++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2087 if (ch >= 256) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2088 if (latin1_encoding_error(&p, &s, errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2089 "ordinal not in range(256)"))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2090 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2091 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2092 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2093 *s++ = (char)ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2094 }
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2095 /* Resize if error handling skipped some characters */
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2096 if (s - start < PyString_GET_SIZE(repr))
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2097 if (_PyString_Resize(&repr, s - start))
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2098 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2099 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2100
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2101 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2102 Py_DECREF(repr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2103 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2104 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2105
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2106 PyObject PyUnicode_AsLatin1String(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2107 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2108 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2109 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2110 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2111 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2112 return PyUnicode_EncodeLatin1(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2113 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2114 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2115 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2116
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2117 /* --- 7-bit ASCII Codec -------------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2118
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2119 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2120 int ascii_decoding_error(const char **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2121 Py_UNICODE **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2122 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2123 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2124 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2125 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2126 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2127 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2128 "ASCII decoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2129 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2130 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2131 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2132 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2133 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2134 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2135 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2136 **dest = Py_UNICODE_REPLACEMENT_CHARACTER;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2137 (*dest)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2138 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2139 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2140 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2141 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2142 "ASCII decoding error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2143 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2144 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2145 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2146 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2147 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2148
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2149 PyObject PyUnicode_DecodeASCII(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2150 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2151 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2152 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2153 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2154 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2155
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2156 /* ASCII is equivalent to the first 128 ordinals in Unicode. */
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2157 if (size == 1 && (unsigned char)s < 128) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2158 Py_UNICODE r = (unsigned char)s;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2159 return PyUnicode_FromUnicode(&r, 1);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2160 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2161
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2162 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2163 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2164 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2165 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2166 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2167 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2168 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2169 register unsigned char c;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2170
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2171 c = (unsigned char)*s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2172 if (c < 128)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2173 *p++ = c;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2174 else if (ascii_decoding_error(&s, &p, errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2175 "ordinal not in range(128)"))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2176 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2177 }
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2178 if (p - PyUnicode_AS_UNICODE(v) < PyString_GET_SIZE(v))
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2179 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2180 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2181 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2182
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2183 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2184 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2185 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2186 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2187
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2188 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2189 int ascii_encoding_error(const Py_UNICODE **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2190 char **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2191 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2192 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2193 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2194 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2195 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2196 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2197 "ASCII encoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2198 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2199 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2200 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2201 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2202 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2203 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2204 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2205 **dest = '?';
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2206 (*dest)++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2207 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2208 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2209 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2210 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2211 "ASCII encoding error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2212 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2213 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2214 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2215 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2216 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2217
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2218 PyObject PyUnicode_EncodeASCII(const Py_UNICODE p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2219 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2220 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2221 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2222 PyObject *repr;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2223 char s, start;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2224
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2225 repr = PyString_FromStringAndSize(NULL, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2226 if (repr == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2227 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2228 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2229 return repr;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2230
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2231 s = PyString_AS_STRING(repr);
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2232 start = s;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2233 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2234 Py_UNICODE ch = *p++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2235 if (ch >= 128) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2236 if (ascii_encoding_error(&p, &s, errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2237 "ordinal not in range(128)"))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2238 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2239 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2240 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2241 *s++ = (char)ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2242 }
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2243 /* Resize if error handling skipped some characters */
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2244 if (s - start < PyString_GET_SIZE(repr))
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2245 if (_PyString_Resize(&repr, s - start))
3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2246 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2247 return repr;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2248
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2249 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2250 Py_DECREF(repr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2251 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2252 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2253
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2254 PyObject PyUnicode_AsASCIIString(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2255 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2256 if (!PyUnicode_Check(unicode)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2257 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2258 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2259 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2260 return PyUnicode_EncodeASCII(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2261 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2262 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2264
18278 b5658a230e1a experimental UCS-4 support: don't assume that MS_WIN32 implies Fredrik Lundh <fredrik@pythonware.com> parents: 18120 diff changeset	2265 #if defined(MS_WIN32) && defined(HAVE_USABLE_WCHAR_T)
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2266
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2267 /* --- MBCS codecs for Windows -------------------------------------------- */
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2268
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2269 PyObject PyUnicode_DecodeMBCS(const char s,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2270 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2271 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2272 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2273 PyUnicodeObject *v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2274 Py_UNICODE *p;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2275
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2276 /* First get the size of the result */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2277 DWORD usize = MultiByteToWideChar(CP_ACP, 0, s, size, NULL, 0);
12582 0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2278 if (size > 0 && usize==0)
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2279 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2280
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2281 v = _PyUnicode_New(usize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2282 if (v == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2283 return NULL;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2284 if (usize == 0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2285 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2286 p = PyUnicode_AS_UNICODE(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2287 if (0 == MultiByteToWideChar(CP_ACP, 0, s, size, p, usize)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2288 Py_DECREF(v);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2289 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2290 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2291
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2292 return (PyObject *)v;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2293 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2294
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2295 PyObject PyUnicode_EncodeMBCS(const Py_UNICODE p,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2296 int size,
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2297 const char *errors)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2298 {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2299 PyObject *repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2300 char *s;
12582 0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2301 DWORD mbcssize;
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2302
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2303 /* If there are no characters, bail now! */
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2304 if (size==0)
0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2305 return PyString_FromString("");
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2306
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2307 /* First get the size of the result */
12582 0bab1e36a3a6 Mark Hammond should get his act into gear (his words :-). Zero length Guido van Rossum <guido@python.org> parents: 12577 diff changeset	2308 mbcssize = WideCharToMultiByte(CP_ACP, 0, p, size, NULL, 0, NULL, NULL);
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2309 if (mbcssize==0)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2310 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2311
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2312 repr = PyString_FromStringAndSize(NULL, mbcssize);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2313 if (repr == NULL)
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2314 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2315 if (mbcssize == 0)
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2316 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2317
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2318 /* Do the conversion */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2319 s = PyString_AS_STRING(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2320 if (0 == WideCharToMultiByte(CP_ACP, 0, p, size, s, mbcssize, NULL, NULL)) {
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2321 Py_DECREF(repr);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2322 return PyErr_SetFromWindowsErrWithFilename(0, NULL);
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2323 }
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2324 return repr;
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2325 }
12193 c3fc2b4617fc Some blank lines. Guido van Rossum <guido@python.org> parents: 12067 diff changeset	2326
12067 bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2327 #endif /* MS_WIN32 */
bcab189ddf86 MBCS codecs. (Win32 only.) By Mark Hammond. Guido van Rossum <guido@python.org> parents: 12025 diff changeset	2328
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2329 /* --- Character Mapping Codec -------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2330
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2331 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2332 int charmap_decoding_error(const char **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2333 Py_UNICODE **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2334 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2335 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2336 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2337 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2338 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2339 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2340 "charmap decoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2341 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2342 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2343 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2344 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2345 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2346 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2347 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2348 **dest = Py_UNICODE_REPLACEMENT_CHARACTER;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2349 (*dest)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2350 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2351 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2352 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2353 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2354 "charmap decoding error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2355 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2356 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2357 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2358 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2359 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2360
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2361 PyObject PyUnicode_DecodeCharmap(const char s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2362 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2363 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2364 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2365 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2366 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2367 Py_UNICODE *p;
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2368 int extrachars = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2369
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2370 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2371 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2372 return PyUnicode_DecodeLatin1(s, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2373
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2374 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2375 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2376 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2377 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2378 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2379 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2380 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2381 unsigned char ch = *s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2382 PyObject w, x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2383
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2384 /* Get mapping (char ordinal -> integer, Unicode char or None) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2385 w = PyInt_FromLong((long)ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2386 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2387 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2388 x = PyObject_GetItem(mapping, w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2389 Py_DECREF(w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2390 if (x == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2391 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2392 /* No mapping found means: mapping is undefined. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2393 PyErr_Clear();
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2394 x = Py_None;
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2395 Py_INCREF(x);
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2396 } else
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2397 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2398 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2399
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2400 /* Apply mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2401 if (PyInt_Check(x)) {
13390 62754b9ec7df Fixed a couple of places where 'int' was used where 'long' Marc-André Lemburg <mal@egenix.com> parents: 13356 diff changeset	2402 long value = PyInt_AS_LONG(x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2403 if (value < 0 \|\| value > 65535) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2404 PyErr_SetString(PyExc_TypeError,
12909 be36f103a953 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12902 diff changeset	2405 "character mapping must be in range(65536)");
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2406 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2407 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2408 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2409 *p++ = (Py_UNICODE)value;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2410 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2411 else if (x == Py_None) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2412 /* undefined mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2413 if (charmap_decoding_error(&s, &p, errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2414 "character maps to <undefined>")) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2415 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2416 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2417 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2418 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2419 else if (PyUnicode_Check(x)) {
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2420 int targetsize = PyUnicode_GET_SIZE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2421
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2422 if (targetsize == 1)
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2423 /* 1-1 mapping */
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2424 p++ = PyUnicode_AS_UNICODE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2425
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2426 else if (targetsize > 1) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2427 /* 1-n mapping */
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2428 if (targetsize > extrachars) {
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2429 /* resize first */
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2430 int oldpos = (int)(p - PyUnicode_AS_UNICODE(v));
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2431 int needed = (targetsize - extrachars) + \
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2432 (targetsize << 2);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2433 extrachars += needed;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2434 if (_PyUnicode_Resize(&v,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2435 PyUnicode_GET_SIZE(v) + needed)) {
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2436 Py_DECREF(x);
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2437 goto onError;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2438 }
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2439 p = PyUnicode_AS_UNICODE(v) + oldpos;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2440 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2441 Py_UNICODE_COPY(p,
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2442 PyUnicode_AS_UNICODE(x),
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2443 targetsize);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2444 p += targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2445 extrachars -= targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2446 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2447 /* 1-0 mapping: skip the character */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2448 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2449 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2450 /* wrong return value */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2451 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2452 "character mapping must return integer, None or unicode");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2453 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2454 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2455 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2456 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2457 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2458 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2459 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2460 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2461 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2462
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2463 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2464 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2465 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2466 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2467
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2468 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2469 int charmap_encoding_error(const Py_UNICODE **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2470 char **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2471 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2472 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2473 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2474 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2475 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2476 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2477 "charmap encoding error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2478 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2479 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2480 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2481 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2482 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2483 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2484 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2485 **dest = '?';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2486 (*dest)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2487 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2488 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2489 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2490 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2491 "charmap encoding error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2492 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2493 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2494 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2495 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2496 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2497
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2498 PyObject PyUnicode_EncodeCharmap(const Py_UNICODE p,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2499 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2500 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2501 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2502 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2503 PyObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2504 char *s;
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2505 int extrachars = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2506
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2507 /* Default to Latin-1 */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2508 if (mapping == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2509 return PyUnicode_EncodeLatin1(p, size, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2510
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2511 v = PyString_FromStringAndSize(NULL, size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2512 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2513 return NULL;
13966 91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2514 if (size == 0)
91ddeed12fa7 Fixed a couple of instances where a 0-length string was being Marc-André Lemburg <mal@egenix.com> parents: 13946 diff changeset	2515 return v;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2516 s = PyString_AS_STRING(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2517 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2518 Py_UNICODE ch = *p++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2519 PyObject w, x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2520
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2521 /* Get mapping (Unicode ordinal -> string char, integer or None) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2522 w = PyInt_FromLong((long)ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2523 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2524 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2525 x = PyObject_GetItem(mapping, w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2526 Py_DECREF(w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2527 if (x == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2528 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2529 /* No mapping found means: mapping is undefined. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2530 PyErr_Clear();
15706 bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2531 x = Py_None;
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2532 Py_INCREF(x);
bf9bb0e3dd34 This patch changes the default behaviour of the builtin charmap Marc-André Lemburg <mal@egenix.com> parents: 15631 diff changeset	2533 } else
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2534 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2535 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2536
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2537 /* Apply mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2538 if (PyInt_Check(x)) {
13390 62754b9ec7df Fixed a couple of places where 'int' was used where 'long' Marc-André Lemburg <mal@egenix.com> parents: 13356 diff changeset	2539 long value = PyInt_AS_LONG(x);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2540 if (value < 0 \|\| value > 255) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2541 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2542 "character mapping must be in range(256)");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2543 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2544 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2545 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2546 *s++ = (char)value;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2547 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2548 else if (x == Py_None) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2549 /* undefined mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2550 if (charmap_encoding_error(&p, &s, errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2551 "character maps to <undefined>")) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2552 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2553 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2554 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2555 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2556 else if (PyString_Check(x)) {
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2557 int targetsize = PyString_GET_SIZE(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2558
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2559 if (targetsize == 1)
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2560 /* 1-1 mapping */
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2561 s++ = PyString_AS_STRING(x);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2562
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2563 else if (targetsize > 1) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2564 /* 1-n mapping */
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2565 if (targetsize > extrachars) {
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2566 /* resize first */
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2567 int oldpos = (int)(s - PyString_AS_STRING(v));
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2568 int needed = (targetsize - extrachars) + \
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2569 (targetsize << 2);
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2570 extrachars += needed;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2571 if (_PyString_Resize(&v, PyString_GET_SIZE(v) + needed)) {
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2572 Py_DECREF(x);
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2573 goto onError;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2574 }
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2575 s = PyString_AS_STRING(v) + oldpos;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2576 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2577 memcpy(s, PyString_AS_STRING(x), targetsize);
15749 77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2578 s += targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2579 extrachars -= targetsize;
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2580 }
77c761cd1272 This patch adds a new feature to the builtin charmap codec: Marc-André Lemburg <mal@egenix.com> parents: 15706 diff changeset	2581 /* 1-0 mapping: skip the character */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2582 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2583 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2584 /* wrong return value */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2585 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2586 "character mapping must return integer, None or unicode");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2587 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2588 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2589 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2590 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2591 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2592 if (s - PyString_AS_STRING(v) < PyString_GET_SIZE(v))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2593 if (_PyString_Resize(&v, (int)(s - PyString_AS_STRING(v))))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2594 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2595 return v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2596
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2597 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2598 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2599 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2600 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2601
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2602 PyObject PyUnicode_AsCharmapString(PyObject unicode,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2603 PyObject *mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2604 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2605 if (!PyUnicode_Check(unicode) \|\| mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2606 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2607 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2608 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2609 return PyUnicode_EncodeCharmap(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2610 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2611 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2612 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2613 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2614
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2615 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2616 int translate_error(const Py_UNICODE **source,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2617 Py_UNICODE **dest,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2618 const char *errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2619 const char *details)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2620 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2621 if ((errors == NULL) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2622 (strcmp(errors,"strict") == 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2623 PyErr_Format(PyExc_UnicodeError,
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2624 "translate error: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2625 details);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2626 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2627 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2628 else if (strcmp(errors,"ignore") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2629 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2630 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2631 else if (strcmp(errors,"replace") == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2632 **dest = '?';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2633 (*dest)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2634 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2635 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2636 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2637 PyErr_Format(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2638 "translate error; "
12320 5de0b939fe61 Skip Montanaro: add string precisions to calls to PyErr_Format Guido van Rossum <guido@python.org> parents: 12275 diff changeset	2639 "unknown error handling code: %.400s",
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2640 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2641 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2642 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2643 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2644
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2645 PyObject PyUnicode_TranslateCharmap(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2646 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2647 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2648 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2649 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2650 PyUnicodeObject *v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2651 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2652
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2653 if (mapping == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2654 PyErr_BadArgument();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2655 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2656 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2657
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2658 /* Output will never be longer than input */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2659 v = _PyUnicode_New(size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2660 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2661 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2662 if (size == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2663 goto done;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2664 p = PyUnicode_AS_UNICODE(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2665 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2666 Py_UNICODE ch = *s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2667 PyObject w, x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2668
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2669 /* Get mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2670 w = PyInt_FromLong(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2671 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2672 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2673 x = PyObject_GetItem(mapping, w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2674 Py_DECREF(w);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2675 if (x == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2676 if (PyErr_ExceptionMatches(PyExc_LookupError)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2677 /* No mapping found: default to 1-1 mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2678 PyErr_Clear();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2679 *p++ = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2680 continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2681 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2682 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2683 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2684
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2685 /* Apply mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2686 if (PyInt_Check(x))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2687 *p++ = (Py_UNICODE)PyInt_AS_LONG(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2688 else if (x == Py_None) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2689 /* undefined mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2690 if (translate_error(&s, &p, errors,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2691 "character maps to <undefined>")) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2692 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2693 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2694 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2695 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2696 else if (PyUnicode_Check(x)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2697 if (PyUnicode_GET_SIZE(x) != 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2698 /* 1-n mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2699 PyErr_SetString(PyExc_NotImplementedError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2700 "1-n mappings are currently not implemented");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2701 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2702 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2703 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2704 p++ = PyUnicode_AS_UNICODE(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2705 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2706 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2707 /* wrong return value */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2708 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2709 "translate mapping must return integer, None or unicode");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2710 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2711 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2712 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2713 Py_DECREF(x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2714 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2715 if (p - PyUnicode_AS_UNICODE(v) < PyUnicode_GET_SIZE(v))
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	2716 if (_PyUnicode_Resize(&v, (int)(p - PyUnicode_AS_UNICODE(v))))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	2717 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2718
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2719 done:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2720 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2721
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2722 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2723 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2724 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2725 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2726
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2727 PyObject PyUnicode_Translate(PyObject str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2728 PyObject *mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2729 const char *errors)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2730 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2731 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2732
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2733 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2734 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2735 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2736 result = PyUnicode_TranslateCharmap(PyUnicode_AS_UNICODE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2737 PyUnicode_GET_SIZE(str),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2738 mapping,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2739 errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2740 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2741 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2742
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2743 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2744 Py_XDECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2745 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2746 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2747
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2748 /* --- Decimal Encoder ---------------------------------------------------- */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2749
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2750 int PyUnicode_EncodeDecimal(Py_UNICODE *s,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2751 int length,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2752 char *output,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2753 const char *errors)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2754 {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2755 Py_UNICODE p, end;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2756
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2757 if (output == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2758 PyErr_BadArgument();
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2759 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2760 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2761
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2762 p = s;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2763 end = s + length;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2764 while (p < end) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2765 register Py_UNICODE ch = *p++;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2766 int decimal;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2767
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2768 if (Py_UNICODE_ISSPACE(ch)) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2769 *output++ = ' ';
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2770 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2771 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2772 decimal = Py_UNICODE_TODECIMAL(ch);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2773 if (decimal >= 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2774 *output++ = '0' + decimal;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2775 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2776 }
12275 cbfa95a6a03e Conrad Huang points out that "if (0 < ch < 256)", while legal C, Guido van Rossum <guido@python.org> parents: 12249 diff changeset	2777 if (0 < ch && ch < 256) {
12577 85f00f8cd674 Fix warning detected by VC++ on assignment of Py_UNICODE to char. Guido van Rossum <guido@python.org> parents: 12576 diff changeset	2778 *output++ = (char)ch;
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2779 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2780 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2781 /* All other characters are considered invalid */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2782 if (errors == NULL \|\| strcmp(errors, "strict") == 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2783 PyErr_SetString(PyExc_ValueError,
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2784 "invalid decimal Unicode string");
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2785 goto onError;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2786 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2787 else if (strcmp(errors, "ignore") == 0)
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2788 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2789 else if (strcmp(errors, "replace") == 0) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2790 *output++ = '?';
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2791 continue;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2792 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2793 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2794 /* 0-terminate the output string */
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2795 *output++ = '\0';
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2796 return 0;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2797
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2798 onError:
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2799 return -1;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2800 }
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	2801
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2802 /* --- Helpers ------------------------------------------------------------ */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2803
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2804 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2805 int count(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2806 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2807 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2808 PyUnicodeObject *substring)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2809 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2810 int count = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2811
15910 f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2812 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2813 start += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2814 if (start < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2815 start = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2816 if (end > self->length)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2817 end = self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2818 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2819 end += self->length;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2820 if (end < 0)
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2821 end = 0;
f7f3be3d00b9 Added checks to prevent PyUnicode_Count() from dumping core Marc-André Lemburg <mal@egenix.com> parents: 15749 diff changeset	2822
12957 97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	2823 if (substring->length == 0)
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	2824 return (end - start + 1);
97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	2825
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2826 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2827
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2828 while (start <= end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2829 if (Py_UNICODE_MATCH(self, start, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2830 count++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2831 start += substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2832 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2833 start++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2834
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2835 return count;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2836 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2837
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2838 int PyUnicode_Count(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2839 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2840 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2841 int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2842 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2843 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2844
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2845 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2846 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2847 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2848 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2849 if (substr == NULL) {
12957 97dd6dc33503 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12939 diff changeset	2850 Py_DECREF(str);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2851 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2852 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2853
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2854 result = count((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2855 start, end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2856 (PyUnicodeObject *)substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2857
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2858 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2859 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2860 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2861 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2862
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2863 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2864 int findstring(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2865 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2866 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2867 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2868 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2869 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2870 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2871 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2872 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2873 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2874
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2875 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2876 return start;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2877
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2878 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2879 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2880 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2881 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2882 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2883 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2884
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2885 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2886
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2887 if (direction < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2888 for (; end >= start; end--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2889 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2890 return end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2891 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2892 for (; start <= end; start++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2893 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2894 return start;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2895 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2896
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2897 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2898 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2899
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2900 int PyUnicode_Find(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2901 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2902 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2903 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2904 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2905 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2906 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2907
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2908 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2909 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2910 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2911 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2912 if (substr == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2913 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2914 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2915 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2916
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2917 result = findstring((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2918 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2919 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2920 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2921 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2922 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2923 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2924
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2925 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2926 int tailmatch(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2927 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2928 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2929 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2930 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2931 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2932 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2933 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2934 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2935 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2936
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2937 if (substring->length == 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2938 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2939
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2940 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2941 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2942 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2943 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2944 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2945 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2946
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2947 end -= substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2948 if (end < start)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2949 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2950
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2951 if (direction > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2952 if (Py_UNICODE_MATCH(self, end, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2953 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2954 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2955 if (Py_UNICODE_MATCH(self, start, substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2956 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2957 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2958
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2959 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2960 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2961
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2962 int PyUnicode_Tailmatch(PyObject *str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2963 PyObject *substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2964 int start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2965 int end,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2966 int direction)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2967 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2968 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2969
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2970 str = PyUnicode_FromObject(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2971 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2972 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2973 substr = PyUnicode_FromObject(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2974 if (substr == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2975 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2976 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2977 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2978
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2979 result = tailmatch((PyUnicodeObject *)str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2980 (PyUnicodeObject *)substr,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2981 start, end, direction);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2982 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2983 Py_DECREF(substr);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2984 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2985 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2986
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2987 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2988 const Py_UNICODE findchar(const Py_UNICODE s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2989 int size,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2990 Py_UNICODE ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2991 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2992 /* like wcschr, but doesn't stop at NULL characters */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2993
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2994 while (size-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2995 if (*s == ch)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2996 return s;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2997 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2998 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	2999
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3000 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3001 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3002
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3003 /* Apply fixfct filter to the Unicode object self and return a
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3004 reference to the modified object */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3005
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3006 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3007 PyObject fixup(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3008 int (fixfct)(PyUnicodeObject s))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3009 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3010
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3011 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3012
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3013 u = (PyUnicodeObject*) PyUnicode_FromUnicode(NULL, self->length);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3014 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3015 return NULL;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3016
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3017 Py_UNICODE_COPY(u->str, self->str, self->length);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3018
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3019 if (!fixfct(u) && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3020 /* fixfct should return TRUE if it modified the buffer. If
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3021 FALSE, return a reference to the original buffer instead
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3022 (to save space, not time) */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3023 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3024 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3025 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3026 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3027 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3028 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3029
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3030 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3031 int fixupper(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3032 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3033 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3034 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3035 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3036
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3037 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3038 register Py_UNICODE ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3039
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3040 ch = Py_UNICODE_TOUPPER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3041 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3042 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3043 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3044 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3045 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3046 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3047
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3048 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3049 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3050
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3051 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3052 int fixlower(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3053 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3054 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3055 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3056 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3057
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3058 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3059 register Py_UNICODE ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3060
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3061 ch = Py_UNICODE_TOLOWER(*s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3062 if (ch != *s) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3063 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3064 *s = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3065 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3066 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3067 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3068
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3069 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3070 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3071
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3072 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3073 int fixswapcase(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3074 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3075 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3076 Py_UNICODE *s = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3077 int status = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3078
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3079 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3080 if (Py_UNICODE_ISUPPER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3081 s = Py_UNICODE_TOLOWER(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3082 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3083 } else if (Py_UNICODE_ISLOWER(*s)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3084 s = Py_UNICODE_TOUPPER(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3085 status = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3086 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3087 s++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3088 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3089
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3090 return status;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3091 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3092
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3093 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3094 int fixcapitalize(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3095 {
16332 bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3096 int len = self->length;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3097 Py_UNICODE *s = self->str;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3098 int status = 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3099
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3100 if (len == 0)
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3101 return 0;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3102 if (Py_UNICODE_ISLOWER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3103 s = Py_UNICODE_TOUPPER(s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3104 status = 1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3105 }
16332 bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3106 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3107 while (--len > 0) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3108 if (Py_UNICODE_ISUPPER(*s)) {
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3109 s = Py_UNICODE_TOLOWER(s);
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3110 status = 1;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3111 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3112 s++;
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3113 }
bacbb45fe844 Fixed .capitalize() method of Unicode objects to work like the Marc-André Lemburg <mal@egenix.com> parents: 16226 diff changeset	3114 return status;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3115 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3116
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3117 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3118 int fixtitle(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3119 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3120 register Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3121 register Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3122 int previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3123
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3124 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3125 if (PyUnicode_GET_SIZE(self) == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3126 Py_UNICODE ch = Py_UNICODE_TOTITLE(*p);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3127 if (*p != ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3128 *p = ch;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3129 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3130 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3131 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3132 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3133 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3134
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3135 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3136 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3137 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3138 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3139
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3140 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3141 *p = Py_UNICODE_TOLOWER(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3142 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3143 *p = Py_UNICODE_TOTITLE(ch);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3144
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3145 if (Py_UNICODE_ISLOWER(ch) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3146 Py_UNICODE_ISUPPER(ch) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3147 Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3148 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3149 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3150 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3151 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3152 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3153 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3154
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3155 PyObject PyUnicode_Join(PyObject separator,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3156 PyObject *seq)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3157 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3158 Py_UNICODE *sep;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3159 int seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3160 PyUnicodeObject *res = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3161 int reslen = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3162 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3163 int sz = 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3164 int i;
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3165 PyObject *it;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3166
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3167 it = PyObject_GetIter(seq);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3168 if (it == NULL)
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3169 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3170
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3171 if (separator == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3172 Py_UNICODE blank = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3173 sep = &blank;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3174 seplen = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3175 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3176 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3177 separator = PyUnicode_FromObject(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3178 if (separator == NULL)
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3179 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3180 sep = PyUnicode_AS_UNICODE(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3181 seplen = PyUnicode_GET_SIZE(separator);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3182 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3183
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3184 res = _PyUnicode_New(sz);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3185 if (res == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3186 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3187 p = PyUnicode_AS_UNICODE(res);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3188 reslen = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3189
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3190 for (i = 0; ; ++i) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3191 int itemlen;
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3192 PyObject *item = PyIter_Next(it);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3193 if (item == NULL) {
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3194 if (PyErr_Occurred())
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3195 goto onError;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3196 break;
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3197 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3198 if (!PyUnicode_Check(item)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3199 PyObject *v;
20007 56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3200 if (!PyString_Check(item)) {
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3201 PyErr_Format(PyExc_TypeError,
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3202 "sequence item %i: expected string or Unicode,"
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3203 " %.80s found",
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3204 i, item->ob_type->tp_name);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3205 Py_DECREF(item);
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3206 goto onError;
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3207 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3208 v = PyUnicode_FromObject(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3209 Py_DECREF(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3210 item = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3211 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3212 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3213 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3214 itemlen = PyUnicode_GET_SIZE(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3215 while (reslen + itemlen + seplen >= sz) {
20007 56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3216 if (_PyUnicode_Resize(&res, sz*2)) {
56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3217 Py_DECREF(item);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3218 goto onError;
20007 56ffee1eeeec Fix Unicode .join() method to raise a TypeError for sequence Marc-André Lemburg <mal@egenix.com> parents: 19999 diff changeset	3219 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3220 sz *= 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3221 p = PyUnicode_AS_UNICODE(res) + reslen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3222 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3223 if (i > 0) {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3224 Py_UNICODE_COPY(p, sep, seplen);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3225 p += seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3226 reslen += seplen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3227 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3228 Py_UNICODE_COPY(p, PyUnicode_AS_UNICODE(item), itemlen);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3229 p += itemlen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3230 reslen += itemlen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3231 Py_DECREF(item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3232 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3233 if (_PyUnicode_Resize(&res, reslen))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3234 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3235
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3236 Py_XDECREF(separator);
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3237 Py_DECREF(it);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3238 return (PyObject *)res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3239
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3240 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3241 Py_XDECREF(separator);
17687 9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3242 Py_XDECREF(res);
9e093bf8f6a9 Make unicode.join() work nice with iterators. This also required a change Tim Peters <tim.peters@gmail.com> parents: 17640 diff changeset	3243 Py_DECREF(it);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3244 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3245 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3246
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3247 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3248 PyUnicodeObject pad(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3249 int left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3250 int right,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3251 Py_UNICODE fill)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3252 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3253 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3254
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3255 if (left < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3256 left = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3257 if (right < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3258 right = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3259
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3260 if (left == 0 && right == 0 && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3261 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3262 return self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3263 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3264
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3265 u = _PyUnicode_New(left + self->length + right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3266 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3267 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3268 Py_UNICODE_FILL(u->str, fill, left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3269 Py_UNICODE_COPY(u->str + left, self->str, self->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3270 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3271 Py_UNICODE_FILL(u->str + left + self->length, fill, right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3272 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3273
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3274 return u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3275 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3276
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3277 #define SPLIT_APPEND(data, left, right) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3278 str = PyUnicode_FromUnicode(data + left, right - left); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3279 if (!str) \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3280 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3281 if (PyList_Append(list, str)) { \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3282 Py_DECREF(str); \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3283 goto onError; \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3284 } \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3285 else \
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3286 Py_DECREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3287
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3288 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3289 PyObject split_whitespace(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3290 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3291 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3292 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3293 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3294 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3295 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3296 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3297
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3298 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3299 /* find a token */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3300 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3301 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3302 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3303 while (i < len && !Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3304 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3305 if (j < i) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3306 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3307 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3308 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3309 while (i < len && Py_UNICODE_ISSPACE(self->str[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3310 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3311 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3312 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3313 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3314 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3315 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3316 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3317 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3318
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3319 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3320 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3321 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3322 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3323
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3324 PyObject PyUnicode_Splitlines(PyObject string,
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	3325 int keepends)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3326 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3327 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3328 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3329 int len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3330 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3331 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3332 Py_UNICODE *data;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3333
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3334 string = PyUnicode_FromObject(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3335 if (string == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3336 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3337 data = PyUnicode_AS_UNICODE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3338 len = PyUnicode_GET_SIZE(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3339
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3340 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3341 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3342 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3343
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3344 for (i = j = 0; i < len; ) {
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	3345 int eol;
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	3346
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3347 /* Find a line and append it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3348 while (i < len && !Py_UNICODE_ISLINEBREAK(data[i]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3349 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3350
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3351 /* Skip the line break reading CRLF as one line break */
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	3352 eol = i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3353 if (i < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3354 if (data[i] == '\r' && i + 1 < len &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3355 data[i+1] == '\n')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3356 i += 2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3357 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3358 i++;
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	3359 if (keepends)
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	3360 eol = i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3361 }
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	3362 SPLIT_APPEND(data, j, eol);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3363 j = i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3364 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3365 if (j < len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3366 SPLIT_APPEND(data, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3367 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3368
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3369 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3370 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3371
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3372 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3373 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3374 Py_DECREF(string);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3375 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3376 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3377
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3378 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3379 PyObject split_char(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3380 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3381 Py_UNICODE ch,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3382 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3383 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3384 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3385 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3386 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3387 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3388
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3389 for (i = j = 0; i < len; ) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3390 if (self->str[i] == ch) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3391 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3392 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3393 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3394 i = j = i + 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3395 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3396 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3397 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3398 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3399 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3400 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3401 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3402
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3403 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3404 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3405 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3406 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3407
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3408 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3409 PyObject split_substring(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3410 PyObject *list,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3411 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3412 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3413 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3414 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3415 register int j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3416 int len = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3417 int sublen = substring->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3418 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3419
15609 0906c8e2c961 Fix off-by-one error in split_substring(). Fixes SF bug #122162. Guido van Rossum <guido@python.org> parents: 15579 diff changeset	3420 for (i = j = 0; i <= len - sublen; ) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3421 if (Py_UNICODE_MATCH(self, i, substring)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3422 if (maxcount-- <= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3423 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3424 SPLIT_APPEND(self->str, j, i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3425 i = j = i + sublen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3426 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3427 i++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3428 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3429 if (j <= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3430 SPLIT_APPEND(self->str, j, len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3431 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3432 return list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3433
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3434 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3435 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3436 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3437 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3438
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3439 #undef SPLIT_APPEND
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3440
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3441 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3442 PyObject split(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3443 PyUnicodeObject *substring,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3444 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3445 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3446 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3447
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3448 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3449 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3450
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3451 list = PyList_New(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3452 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3453 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3454
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3455 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3456 return split_whitespace(self,list,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3457
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3458 else if (substring->length == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3459 return split_char(self,list,substring->str[0],maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3460
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3461 else if (substring->length == 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3462 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3463 PyErr_SetString(PyExc_ValueError, "empty separator");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3464 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3465 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3466 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3467 return split_substring(self,list,substring,maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3468 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3469
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3470 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3471 PyObject strip(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3472 int left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3473 int right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3474 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3475 Py_UNICODE *p = self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3476 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3477 int end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3478
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3479 if (left)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3480 while (start < end && Py_UNICODE_ISSPACE(p[start]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3481 start++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3482
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3483 if (right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3484 while (end > start && Py_UNICODE_ISSPACE(p[end-1]))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3485 end--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3486
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3487 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3488 /* couldn't strip anything off, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3489 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3490 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3491 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3492
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3493 return (PyObject*) PyUnicode_FromUnicode(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3494 self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3495 end - start
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3496 );
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3497 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3498
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3499 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3500 PyObject replace(PyUnicodeObject self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3501 PyUnicodeObject *str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3502 PyUnicodeObject *str2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3503 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3504 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3505 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3506
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3507 if (maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3508 maxcount = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3509
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3510 if (str1->length == 1 && str2->length == 1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3511 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3512
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3513 /* replace characters */
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3514 if (!findchar(self->str, self->length, str1->str[0]) &&
72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3515 PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3516 /* nothing to replace, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3517 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3518 u = self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3519 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3520 Py_UNICODE u1 = str1->str[0];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3521 Py_UNICODE u2 = str2->str[0];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3522
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3523 u = (PyUnicodeObject*) PyUnicode_FromUnicode(
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3524 NULL,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3525 self->length
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3526 );
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3527 if (u != NULL) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3528 Py_UNICODE_COPY(u->str, self->str,
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3529 self->length);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3530 for (i = 0; i < u->length; i++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3531 if (u->str[i] == u1) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3532 if (--maxcount < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3533 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3534 u->str[i] = u2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3535 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3536 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	3537 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3538
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3539 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3540 int n, i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3541 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3542
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3543 /* replace strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3544 n = count(self, 0, self->length, str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3545 if (n > maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3546 n = maxcount;
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3547 if (n == 0 && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3548 /* nothing to replace, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3549 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3550 u = self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3551 } else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3552 u = _PyUnicode_New(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3553 self->length + n * (str2->length - str1->length));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3554 if (u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3555 i = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3556 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3557 while (i <= self->length - str1->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3558 if (Py_UNICODE_MATCH(self, i, str1)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3559 /* replace string segment */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3560 Py_UNICODE_COPY(p, str2->str, str2->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3561 p += str2->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3562 i += str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3563 if (--n <= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3564 /* copy remaining part */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3565 Py_UNICODE_COPY(p, self->str+i, self->length-i);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3566 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3567 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3568 } else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3569 *p++ = self->str[i++];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3570 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3571 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3572 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3573
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3574 return (PyObject *) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3575 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3576
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3577 /* --- Unicode Object Methods --------------------------------------------- */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3578
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3579 static char title__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3580 "S.title() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3581 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3582 Return a titlecased version of S, i.e. words start with title case\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3583 characters, all remaining cased characters have lower case.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3584
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3585 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	3586 unicode_title(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3587 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3588 return fixup(self, fixtitle);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3589 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3590
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3591 static char capitalize__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3592 "S.capitalize() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3593 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3594 Return a capitalized version of S, i.e. make the first character\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3595 have upper case.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3596
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3597 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	3598 unicode_capitalize(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3599 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3600 return fixup(self, fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3601 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3602
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3603 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3604 static char capwords__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3605 "S.capwords() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3606 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3607 Apply .capitalize() to all words in S and return the result with\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3608 normalized whitespace (all whitespace strings are replaced by ' ').";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3609
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3610 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	3611 unicode_capwords(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3612 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3613 PyObject *list;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3614 PyObject *item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3615 int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3616
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3617 /* Split into words */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3618 list = split(self, NULL, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3619 if (!list)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3620 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3621
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3622 /* Capitalize each word */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3623 for (i = 0; i < PyList_GET_SIZE(list); i++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3624 item = fixup((PyUnicodeObject *)PyList_GET_ITEM(list, i),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3625 fixcapitalize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3626 if (item == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3627 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3628 Py_DECREF(PyList_GET_ITEM(list, i));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3629 PyList_SET_ITEM(list, i, item);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3630 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3631
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3632 /* Join the words to form a new string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3633 item = PyUnicode_Join(NULL, list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3634
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3635 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3636 Py_DECREF(list);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3637 return (PyObject *)item;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3638 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3639 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3640
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3641 static char center__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3642 "S.center(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3643 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3644 Return S centered in a Unicode string of length width. Padding is done\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3645 using spaces.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3646
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3647 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3648 unicode_center(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3649 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3650 int marg, left;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3651 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3652
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3653 if (!PyArg_ParseTuple(args, "i:center", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3654 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3655
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	3656 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3657 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3658 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3659 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3660
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3661 marg = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3662 left = marg / 2 + (marg & width & 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3663
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3664 return (PyObject*) pad(self, left, marg - left, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3665 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3666
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3667 #if 0
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3668
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3669 /* This code should go into some future Unicode collation support
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3670 module. The basic comparison should compare ordinals on a naive
13946 7a99450da2d6 Clean up warning from Monterey compiler. Trent Mick <trentm@activestate.com> parents: 13890 diff changeset	3671 basis (this is what Java does and thus JPython too). */
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3672
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3673 /* speedy UTF-16 code point order comparison */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3674 /* gleaned from: */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3675 /* http://www-4.ibm.com/software/developer/library/utf16.html?dwzone=unicode */
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3676
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	3677 static short utf16Fixup[32] =
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3678 {
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3679 0, 0, 0, 0, 0, 0, 0, 0,
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3680 0, 0, 0, 0, 0, 0, 0, 0,
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3681 0, 0, 0, 0, 0, 0, 0, 0,
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	3682 0, 0, 0, 0x2000, -0x800, -0x800, -0x800, -0x800
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3683 };
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3684
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3685 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3686 unicode_compare(PyUnicodeObject str1, PyUnicodeObject str2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3687 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3688 int len1, len2;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3689
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3690 Py_UNICODE *s1 = str1->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3691 Py_UNICODE *s2 = str2->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3692
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3693 len1 = str1->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3694 len2 = str2->length;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3695
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3696 while (len1 > 0 && len2 > 0) {
13411 9a8391054840 New surrogate support in the UTF-8 codec. By Bill Tutt. Marc-André Lemburg <mal@egenix.com> parents: 13401 diff changeset	3697 Py_UNICODE c1, c2;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3698
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3699 c1 = *s1++;
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3700 c2 = *s2++;
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3701
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3702 if (c1 > (1<<11) * 26)
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3703 c1 += utf16Fixup[c1>>11];
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3704 if (c2 > (1<<11) * 26)
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3705 c2 += utf16Fixup[c2>>11];
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3706 /* now c1 and c2 are in UTF-32-compatible order */
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3707
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3708 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3709 return (c1 < c2) ? -1 : 1;
13337 e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3710
e101600adee4 Bill Tutt: Marc-André Lemburg <mal@egenix.com> parents: 13209 diff changeset	3711 len1--; len2--;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3712 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3713
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3714 return (len1 < len2) ? -1 : (len1 != len2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3715 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3716
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3717 #else
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3718
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3719 static int
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3720 unicode_compare(PyUnicodeObject str1, PyUnicodeObject str2)
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3721 {
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3722 register int len1, len2;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3723
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3724 Py_UNICODE *s1 = str1->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3725 Py_UNICODE *s2 = str2->str;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3726
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3727 len1 = str1->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3728 len2 = str2->length;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3729
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3730 while (len1 > 0 && len2 > 0) {
18279 8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3731 Py_UNICODE c1, c2;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3732
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3733 c1 = *s1++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3734 c2 = *s2++;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3735
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3736 if (c1 != c2)
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3737 return (c1 < c2) ? -1 : 1;
8b7ff44b4669 experimental UCS-4 support: made compare a bit more robust, in case Fredrik Lundh <fredrik@pythonware.com> parents: 18278 diff changeset	3738
13890 8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3739 len1--; len2--;
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3740 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3741
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3742 return (len1 < len2) ? -1 : (len1 != len2);
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3743 }
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3744
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3745 #endif
8f0fc0b5f784 Removing UTF-16 aware Unicode comparison code. This kind of compare Marc-André Lemburg <mal@egenix.com> parents: 13837 diff changeset	3746
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3747 int PyUnicode_Compare(PyObject *left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3748 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3749 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3750 PyUnicodeObject u = NULL, v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3751 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3752
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3753 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3754 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3755 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3756 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3757 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3758 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3759 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3760
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 13576 diff changeset	3761 /* Shortcut for empty or interned objects */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3762 if (v == u) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3763 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3764 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3765 return 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3766 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3767
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3768 result = unicode_compare(u, v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3769
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3770 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3771 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3772 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3773
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3774 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3775 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3776 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3777 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3778 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3779
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3780 int PyUnicode_Contains(PyObject *container,
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3781 PyObject *element)
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3782 {
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3783 PyUnicodeObject u = NULL, v = NULL;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3784 int result;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3785 register const Py_UNICODE p, e;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3786 register Py_UNICODE ch;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3787
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3788 /* Coerce the two arguments */
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3789 v = (PyUnicodeObject *)PyUnicode_FromObject(element);
13049 0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	3790 if (v == NULL) {
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	3791 PyErr_SetString(PyExc_TypeError,
0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	3792 "'in <string>' requires character as left operand");
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3793 goto onError;
13049 0be4b7b650e3 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12957 diff changeset	3794 }
12248 70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3795 u = (PyUnicodeObject *)PyUnicode_FromObject(container);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3796 if (u == NULL) {
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3797 Py_DECREF(v);
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3798 goto onError;
70154e2e9c89 Marc-Andre's third try at this bulk patch seems to work (except that Guido van Rossum <guido@python.org> parents: 12193 diff changeset	3799 }
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3800
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3801 /* Check v in u */
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3802 if (PyUnicode_GET_SIZE(v) != 1) {
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3803 PyErr_SetString(PyExc_TypeError,
12902 98eb3059ae76 Patch from Michael Hudson: improve unclear error message Andrew M. Kuchling <amk@amk.ca> parents: 12899 diff changeset	3804 "'in <string>' requires character as left operand");
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3805 goto onError;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3806 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3807 ch = *PyUnicode_AS_UNICODE(v);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3808 p = PyUnicode_AS_UNICODE(u);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3809 e = p + PyUnicode_GET_SIZE(u);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3810 result = 0;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3811 while (p < e) {
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3812 if (*p++ == ch) {
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3813 result = 1;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3814 break;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3815 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3816 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3817
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3818 Py_DECREF(u);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3819 Py_DECREF(v);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3820 return result;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3821
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3822 onError:
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3823 Py_XDECREF(u);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3824 Py_XDECREF(v);
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3825 return -1;
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3826 }
38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	3827
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3828 /* Concat to string or Unicode object giving a new Unicode object. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3829
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3830 PyObject PyUnicode_Concat(PyObject left,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3831 PyObject *right)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3832 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3833 PyUnicodeObject u = NULL, v = NULL, *w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3834
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3835 /* Coerce the two arguments */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3836 u = (PyUnicodeObject *)PyUnicode_FromObject(left);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3837 if (u == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3838 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3839 v = (PyUnicodeObject *)PyUnicode_FromObject(right);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3840 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3841 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3842
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3843 /* Shortcuts */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3844 if (v == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3845 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3846 return (PyObject *)u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3847 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3848 if (u == unicode_empty) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3849 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3850 return (PyObject *)v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3851 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3852
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3853 /* Concat the two Unicode strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3854 w = _PyUnicode_New(u->length + v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3855 if (w == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3856 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3857 Py_UNICODE_COPY(w->str, u->str, u->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3858 Py_UNICODE_COPY(w->str + u->length, v->str, v->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3859
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3860 Py_DECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3861 Py_DECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3862 return (PyObject *)w;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3863
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3864 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3865 Py_XDECREF(u);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3866 Py_XDECREF(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3867 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3868 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3869
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3870 static char count__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3871 "S.count(sub[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3872 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3873 Return the number of occurrences of substring sub in Unicode string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3874 S[start:end]. Optional arguments start and end are\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3875 interpreted as in slice notation.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3876
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3877 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3878 unicode_count(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3879 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3880 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3881 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3882 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3883 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3884
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	3885 if (!PyArg_ParseTuple(args, "O\|O&O&:count", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	3886 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3887 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3888
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3889 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3890 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3891 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3892 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3893
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3894 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3895 start += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3896 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3897 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3898 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3899 end = self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3900 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3901 end += self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3902 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3903 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3904
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3905 result = PyInt_FromLong((long) count(self, start, end, substring));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3906
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3907 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3908 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3909 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3910
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3911 static char encode__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3912 "S.encode([encoding[,errors]]) -> string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3913 \n\
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	3914 Return an encoded string version of S. Default encoding is the current\n\
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	3915 default string encoding. errors may be given to set a different error\n\
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	3916 handling scheme. Default is 'strict' meaning that encoding errors raise\n\
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	3917 a ValueError. Other possible values are 'ignore' and 'replace'.";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3918
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3919 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3920 unicode_encode(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3921 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3922 char *encoding = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3923 char *errors = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3924 if (!PyArg_ParseTuple(args, "\|ss:encode", &encoding, &errors))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3925 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3926 return PyUnicode_AsEncodedString((PyObject *)self, encoding, errors);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3927 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3928
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3929 static char expandtabs__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3930 "S.expandtabs([tabsize]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3931 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3932 Return a copy of S where all tab characters are expanded using spaces.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3933 If tabsize is not given, a tab size of 8 characters is assumed.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3934
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3935 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3936 unicode_expandtabs(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3937 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3938 Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3939 Py_UNICODE *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3940 Py_UNICODE *q;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3941 int i, j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3942 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3943 int tabsize = 8;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3944
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3945 if (!PyArg_ParseTuple(args, "\|i:expandtabs", &tabsize))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3946 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3947
13620 5b727ab37092 Spelling fixes supplied by Rob W. W. Hooft. All these are fixes in either Thomas Wouters <thomas@python.org> parents: 13576 diff changeset	3948 /* First pass: determine size of output string */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3949 i = j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3950 e = self->str + self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3951 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3952 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3953 if (tabsize > 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3954 j += tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3955 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3956 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3957 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3958 if (p == '\n' \|\| p == '\r') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3959 i += j;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3960 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3961 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3962 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3963
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3964 /* Second pass: create output string and fill it */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3965 u = _PyUnicode_New(i + j);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3966 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3967 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3968
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3969 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3970 q = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3971
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3972 for (p = self->str; p < e; p++)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3973 if (*p == '\t') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3974 if (tabsize > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3975 i = tabsize - (j % tabsize);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3976 j += i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3977 while (i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3978 *q++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3979 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3980 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3981 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3982 j++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3983 q++ = p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3984 if (p == '\n' \|\| p == '\r')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3985 j = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3986 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3987
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3988 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3989 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3990
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3991 static char find__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3992 "S.find(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3993 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3994 Return the lowest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3995 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3996 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3997 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3998 Return -1 on failure.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	3999
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4000 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4001 unicode_find(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4002 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4003 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4004 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4005 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4006 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4007
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4008 if (!PyArg_ParseTuple(args, "O\|O&O&:find", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4009 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4010 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4011 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4012 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4013 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4014 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4015
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4016 result = PyInt_FromLong(findstring(self, substring, start, end, 1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4017
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4018 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4019 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4020 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4021
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4022 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4023 unicode_getitem(PyUnicodeObject *self, int index)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4024 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4025 if (index < 0 \|\| index >= self->length) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4026 PyErr_SetString(PyExc_IndexError, "string index out of range");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4027 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4028 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4029
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4030 return (PyObject*) PyUnicode_FromUnicode(&self->str[index], 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4031 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4032
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4033 static long
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4034 unicode_hash(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4035 {
13522 acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4036 /* Since Unicode objects compare equal to their ASCII string
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4037 counterparts, they should use the individual character values
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4038 as basis for their hash value. This is needed to assure that
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4039 strings and Unicode objects behave in the same way as
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4040 dictionary keys. */
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4041
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4042 register int len;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4043 register Py_UNICODE *p;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4044 register long x;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4045
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4046 if (self->hash != -1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4047 return self->hash;
13522 acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4048 len = PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4049 p = PyUnicode_AS_UNICODE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4050 x = *p << 7;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4051 while (--len >= 0)
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4052 x = (1000003x) ^ p++;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4053 x ^= PyUnicode_GET_SIZE(self);
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4054 if (x == -1)
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4055 x = -2;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4056 self->hash = x;
acc4638f37d5 - changed hash calculation for unicode strings. the new Fredrik Lundh <fredrik@pythonware.com> parents: 13411 diff changeset	4057 return x;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4058 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4059
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4060 static char index__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4061 "S.index(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4062 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4063 Like S.find() but raise ValueError when the substring is not found.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4064
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4065 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4066 unicode_index(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4067 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4068 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4069 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4070 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4071 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4072
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4073 if (!PyArg_ParseTuple(args, "O\|O&O&:index", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4074 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4075 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4076
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4077 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4078 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4079 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4080 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4081
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4082 result = findstring(self, substring, start, end, 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4083
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4084 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4085 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4086 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4087 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4088 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4089 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4090 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4091
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4092 static char islower__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4093 "S.islower() -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4094 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4095 Return 1 if all cased characters in S are lowercase and there is\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4096 at least one cased character in S, 0 otherwise.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4097
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4098 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4099 unicode_islower(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4100 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4101 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4102 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4103 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4104
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4105 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4106 if (PyUnicode_GET_SIZE(self) == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4107 return PyInt_FromLong(Py_UNICODE_ISLOWER(*p) != 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4108
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4109 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4110 if (PyString_GET_SIZE(self) == 0)
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4111 return PyInt_FromLong(0);
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4112
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4113 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4114 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4115 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4116 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4117
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4118 if (Py_UNICODE_ISUPPER(ch) \|\| Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4119 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4120 else if (!cased && Py_UNICODE_ISLOWER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4121 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4122 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4123 return PyInt_FromLong(cased);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4124 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4125
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4126 static char isupper__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4127 "S.isupper() -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4128 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4129 Return 1 if all cased characters in S are uppercase and there is\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4130 at least one cased character in S, 0 otherwise.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4131
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4132 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4133 unicode_isupper(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4134 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4135 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4136 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4137 int cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4138
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4139 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4140 if (PyUnicode_GET_SIZE(self) == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4141 return PyInt_FromLong(Py_UNICODE_ISUPPER(*p) != 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4142
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4143 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4144 if (PyString_GET_SIZE(self) == 0)
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4145 return PyInt_FromLong(0);
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4146
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4147 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4148 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4149 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4150 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4151
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4152 if (Py_UNICODE_ISLOWER(ch) \|\| Py_UNICODE_ISTITLE(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4153 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4154 else if (!cased && Py_UNICODE_ISUPPER(ch))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4155 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4156 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4157 return PyInt_FromLong(cased);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4158 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4159
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4160 static char istitle__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4161 "S.istitle() -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4162 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4163 Return 1 if S is a titlecased string, i.e. upper- and titlecase characters\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4164 may only follow uncased characters and lowercase characters only cased\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4165 ones. Return 0 otherwise.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4166
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4167 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4168 unicode_istitle(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4169 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4170 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4171 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4172 int cased, previous_is_cased;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4173
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4174 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4175 if (PyUnicode_GET_SIZE(self) == 1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4176 return PyInt_FromLong((Py_UNICODE_ISTITLE(*p) != 0) \|\|
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4177 (Py_UNICODE_ISUPPER(*p) != 0));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4178
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4179 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4180 if (PyString_GET_SIZE(self) == 0)
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4181 return PyInt_FromLong(0);
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4182
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4183 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4184 cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4185 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4186 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4187 register const Py_UNICODE ch = *p;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4188
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4189 if (Py_UNICODE_ISUPPER(ch) \|\| Py_UNICODE_ISTITLE(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4190 if (previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4191 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4192 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4193 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4194 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4195 else if (Py_UNICODE_ISLOWER(ch)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4196 if (!previous_is_cased)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4197 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4198 previous_is_cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4199 cased = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4200 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4201 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4202 previous_is_cased = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4203 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4204 return PyInt_FromLong(cased);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4205 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4206
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4207 static char isspace__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4208 "S.isspace() -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4209 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4210 Return 1 if there are only whitespace characters in S,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4211 0 otherwise.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4212
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4213 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4214 unicode_isspace(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4215 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4216 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4217 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4218
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4219 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4220 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4221 Py_UNICODE_ISSPACE(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4222 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4223
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4224 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4225 if (PyString_GET_SIZE(self) == 0)
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4226 return PyInt_FromLong(0);
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4227
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4228 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4229 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4230 if (!Py_UNICODE_ISSPACE(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4231 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4232 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4233 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4234 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4235
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4236 static char isalpha__doc__[] =
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4237 "S.isalpha() -> int\n\
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4238 \n\
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4239 Return 1 if all characters in S are alphabetic\n\
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4240 and there is at least one character in S, 0 otherwise.";
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4241
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4242 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4243 unicode_isalpha(PyUnicodeObject *self)
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4244 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4245 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4246 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4247
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4248 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4249 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4250 Py_UNICODE_ISALPHA(*p))
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4251 return PyInt_FromLong(1);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4252
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4253 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4254 if (PyString_GET_SIZE(self) == 0)
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4255 return PyInt_FromLong(0);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4256
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4257 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4258 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4259 if (!Py_UNICODE_ISALPHA(*p))
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4260 return PyInt_FromLong(0);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4261 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4262 return PyInt_FromLong(1);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4263 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4264
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4265 static char isalnum__doc__[] =
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4266 "S.isalnum() -> int\n\
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4267 \n\
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4268 Return 1 if all characters in S are alphanumeric\n\
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4269 and there is at least one character in S, 0 otherwise.";
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4270
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4271 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4272 unicode_isalnum(PyUnicodeObject *self)
13356 9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4273 {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4274 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4275 register const Py_UNICODE *e;
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4276
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4277 /* Shortcut for single character strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4278 if (PyUnicode_GET_SIZE(self) == 1 &&
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4279 Py_UNICODE_ISALNUM(*p))
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4280 return PyInt_FromLong(1);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4281
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4282 /* Special case for empty strings */
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4283 if (PyString_GET_SIZE(self) == 0)
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4284 return PyInt_FromLong(0);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4285
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4286 e = p + PyUnicode_GET_SIZE(self);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4287 for (; p < e; p++) {
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4288 if (!Py_UNICODE_ISALNUM(*p))
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4289 return PyInt_FromLong(0);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4290 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4291 return PyInt_FromLong(1);
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4292 }
9a1b961e0c3a Added new .isalpha() and .isalnum() methods which provide interfaces Marc-André Lemburg <mal@egenix.com> parents: 13337 diff changeset	4293
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4294 static char isdecimal__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4295 "S.isdecimal() -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4296 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4297 Return 1 if there are only decimal characters in S,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4298 0 otherwise.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4299
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4300 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4301 unicode_isdecimal(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4302 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4303 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4304 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4305
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4306 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4307 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4308 Py_UNICODE_ISDECIMAL(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4309 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4310
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4311 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4312 if (PyString_GET_SIZE(self) == 0)
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4313 return PyInt_FromLong(0);
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4314
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4315 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4316 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4317 if (!Py_UNICODE_ISDECIMAL(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4318 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4319 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4320 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4321 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4322
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4323 static char isdigit__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4324 "S.isdigit() -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4325 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4326 Return 1 if there are only digit characters in S,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4327 0 otherwise.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4328
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4329 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4330 unicode_isdigit(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4331 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4332 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4333 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4334
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4335 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4336 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4337 Py_UNICODE_ISDIGIT(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4338 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4339
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4340 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4341 if (PyString_GET_SIZE(self) == 0)
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4342 return PyInt_FromLong(0);
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4343
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4344 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4345 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4346 if (!Py_UNICODE_ISDIGIT(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4347 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4348 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4349 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4350 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4351
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4352 static char isnumeric__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4353 "S.isnumeric() -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4354 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4355 Return 1 if there are only numeric characters in S,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4356 0 otherwise.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4357
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4358 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4359 unicode_isnumeric(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4360 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4361 register const Py_UNICODE *p = PyUnicode_AS_UNICODE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4362 register const Py_UNICODE *e;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4363
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4364 /* Shortcut for single character strings */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4365 if (PyUnicode_GET_SIZE(self) == 1 &&
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4366 Py_UNICODE_ISNUMERIC(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4367 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4368
12924 2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4369 /* Special case for empty strings */
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4370 if (PyString_GET_SIZE(self) == 0)
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4371 return PyInt_FromLong(0);
2d0b99778edf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12909 diff changeset	4372
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4373 e = p + PyUnicode_GET_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4374 for (; p < e; p++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4375 if (!Py_UNICODE_ISNUMERIC(*p))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4376 return PyInt_FromLong(0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4377 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4378 return PyInt_FromLong(1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4379 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4380
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4381 static char join__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4382 "S.join(sequence) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4383 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4384 Return a string which is the concatenation of the strings in the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4385 sequence. The separator between elements is S.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4386
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4387 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4388 unicode_join(PyObject self, PyObject data)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4389 {
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4390 return PyUnicode_Join(self, data);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4391 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4392
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4393 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4394 unicode_length(PyUnicodeObject *self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4395 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4396 return self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4397 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4398
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4399 static char ljust__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4400 "S.ljust(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4401 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4402 Return S left justified in a Unicode string of length width. Padding is\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4403 done using spaces.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4404
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4405 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4406 unicode_ljust(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4407 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4408 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4409 if (!PyArg_ParseTuple(args, "i:ljust", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4410 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4411
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4412 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4413 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4414 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4415 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4416
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4417 return (PyObject*) pad(self, 0, width - self->length, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4418 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4419
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4420 static char lower__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4421 "S.lower() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4422 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4423 Return a copy of the string S converted to lowercase.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4424
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4425 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4426 unicode_lower(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4427 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4428 return fixup(self, fixlower);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4429 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4430
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4431 static char lstrip__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4432 "S.lstrip() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4433 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4434 Return a copy of the string S with leading whitespace removed.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4435
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4436 static PyObject *
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4437 unicode_lstrip(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4438 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4439 return strip(self, 1, 0);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4440 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4441
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4442 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4443 unicode_repeat(PyUnicodeObject *str, int len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4444 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4445 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4446 Py_UNICODE *p;
14492 7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4447 int nchars;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4448 size_t nbytes;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4449
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4450 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4451 len = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4452
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4453 if (len == 1 && PyUnicode_CheckExact(str)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4454 /* no repeat, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4455 Py_INCREF(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4456 return (PyObject*) str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4457 }
14492 7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4458
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4459 /* ensure # of chars needed doesn't overflow int and # of bytes
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4460 * needed doesn't overflow size_t
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4461 */
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4462 nchars = len * str->length;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4463 if (len && nchars / len != str->length) {
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4464 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4465 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4466 return NULL;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4467 }
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4468 nbytes = (nchars + 1) * sizeof(Py_UNICODE);
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4469 if (nbytes / sizeof(Py_UNICODE) != (size_t)(nchars + 1)) {
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4470 PyErr_SetString(PyExc_OverflowError,
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4471 "repeated string is too long");
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4472 return NULL;
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4473 }
7728b0ea9344 Fix for bug 113934. stringn and unicoden did no overflow checking at Tim Peters <tim.peters@gmail.com> parents: 14388 diff changeset	4474 u = _PyUnicode_New(nchars);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4475 if (!u)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4476 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4477
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4478 p = u->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4479
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4480 while (len-- > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4481 Py_UNICODE_COPY(p, str->str, str->length);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4482 p += str->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4483 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4484
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4485 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4486 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4487
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4488 PyObject PyUnicode_Replace(PyObject obj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4489 PyObject *subobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4490 PyObject *replobj,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4491 int maxcount)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4492 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4493 PyObject *self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4494 PyObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4495 PyObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4496 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4497
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4498 self = PyUnicode_FromObject(obj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4499 if (self == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4500 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4501 str1 = PyUnicode_FromObject(subobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4502 if (str1 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4503 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4504 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4505 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4506 str2 = PyUnicode_FromObject(replobj);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4507 if (str2 == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4508 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4509 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4510 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4511 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4512 result = replace((PyUnicodeObject *)self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4513 (PyUnicodeObject *)str1,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4514 (PyUnicodeObject *)str2,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4515 maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4516 Py_DECREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4517 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4518 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4519 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4520 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4521
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4522 static char replace__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4523 "S.replace (old, new[, maxsplit]) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4524 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4525 Return a copy of S with all occurrences of substring\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4526 old replaced by new. If the optional argument maxsplit is\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4527 given, only the first maxsplit occurrences are replaced.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4528
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4529 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4530 unicode_replace(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4531 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4532 PyUnicodeObject *str1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4533 PyUnicodeObject *str2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4534 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4535 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4536
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4537 if (!PyArg_ParseTuple(args, "OO\|i:replace", &str1, &str2, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4538 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4539 str1 = (PyUnicodeObject )PyUnicode_FromObject((PyObject )str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4540 if (str1 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4541 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4542 str2 = (PyUnicodeObject )PyUnicode_FromObject((PyObject )str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4543 if (str2 == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4544 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4545
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4546 result = replace(self, str1, str2, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4547
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4548 Py_DECREF(str1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4549 Py_DECREF(str2);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4550 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4551 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4552
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4553 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4554 PyObject unicode_repr(PyObject unicode)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4555 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4556 return unicodeescape_string(PyUnicode_AS_UNICODE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4557 PyUnicode_GET_SIZE(unicode),
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4558 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4559 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4560
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4561 static char rfind__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4562 "S.rfind(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4563 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4564 Return the highest index in S where substring sub is found,\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4565 such that sub is contained within s[start,end]. Optional\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4566 arguments start and end are interpreted as in slice notation.\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4567 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4568 Return -1 on failure.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4569
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4570 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4571 unicode_rfind(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4572 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4573 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4574 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4575 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4576 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4577
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4578 if (!PyArg_ParseTuple(args, "O\|O&O&:rfind", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4579 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4580 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4581 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4582 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4583 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4584 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4585
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4586 result = PyInt_FromLong(findstring(self, substring, start, end, -1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4587
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4588 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4589 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4590 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4591
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4592 static char rindex__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4593 "S.rindex(sub [,start [,end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4594 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4595 Like S.rfind() but raise ValueError when the substring is not found.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4596
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4597 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4598 unicode_rindex(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4599 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4600 int result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4601 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4602 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4603 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4604
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4605 if (!PyArg_ParseTuple(args, "O\|O&O&:rindex", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4606 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4607 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4608 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4609 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4610 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4611 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4612
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4613 result = findstring(self, substring, start, end, -1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4614
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4615 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4616 if (result < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4617 PyErr_SetString(PyExc_ValueError, "substring not found");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4618 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4619 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4620 return PyInt_FromLong(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4621 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4622
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4623 static char rjust__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4624 "S.rjust(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4625 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4626 Return S right justified in a Unicode string of length width. Padding is\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4627 done using spaces.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4628
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4629 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4630 unicode_rjust(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4631 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4632 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4633 if (!PyArg_ParseTuple(args, "i:rjust", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4634 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4635
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4636 if (self->length >= width && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4637 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4638 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4639 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4640
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4641 return (PyObject*) pad(self, width - self->length, 0, ' ');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4642 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4643
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4644 static char rstrip__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4645 "S.rstrip() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4646 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4647 Return a copy of the string S with trailing whitespace removed.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4648
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4649 static PyObject *
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4650 unicode_rstrip(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4651 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4652 return strip(self, 0, 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4653 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4654
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4655 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4656 unicode_slice(PyUnicodeObject *self, int start, int end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4657 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4658 /* standard clamping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4659 if (start < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4660 start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4661 if (end < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4662 end = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4663 if (end > self->length)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4664 end = self->length;
19885 72e6eb653312 More on bug 460020: disable many optimizations of unicode subclasses. Tim Peters <tim.peters@gmail.com> parents: 19849 diff changeset	4665 if (start == 0 && end == self->length && PyUnicode_CheckExact(self)) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4666 /* full slice, return original string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4667 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4668 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4669 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4670 if (start > end)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4671 start = end;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4672 /* copy slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4673 return (PyObject*) PyUnicode_FromUnicode(self->str + start,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4674 end - start);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4675 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4676
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4677 PyObject PyUnicode_Split(PyObject s,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4678 PyObject *sep,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4679 int maxsplit)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4680 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4681 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4682
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4683 s = PyUnicode_FromObject(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4684 if (s == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4685 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4686 if (sep != NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4687 sep = PyUnicode_FromObject(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4688 if (sep == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4689 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4690 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4691 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4692 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4693
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4694 result = split((PyUnicodeObject )s, (PyUnicodeObject )sep, maxsplit);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4695
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4696 Py_DECREF(s);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4697 Py_XDECREF(sep);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4698 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4699 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4700
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4701 static char split__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4702 "S.split([sep [,maxsplit]]) -> list of strings\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4703 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4704 Return a list of the words in S, using sep as the\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4705 delimiter string. If maxsplit is given, at most maxsplit\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4706 splits are done. If sep is not specified, any whitespace string\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4707 is a separator.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4708
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4709 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4710 unicode_split(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4711 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4712 PyObject *substring = Py_None;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4713 int maxcount = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4714
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4715 if (!PyArg_ParseTuple(args, "\|Oi:split", &substring, &maxcount))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4716 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4717
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4718 if (substring == Py_None)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4719 return split(self, NULL, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4720 else if (PyUnicode_Check(substring))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4721 return split(self, (PyUnicodeObject *)substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4722 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4723 return PyUnicode_Split((PyObject *)self, substring, maxcount);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4724 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4725
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4726 static char splitlines__doc__[] =
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4727 "S.splitlines([keepends]]) -> list of strings\n\
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4728 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4729 Return a list of the lines in S, breaking at line boundaries.\n\
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4730 Line breaks are not included in the resulting list unless keepends\n\
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4731 is given and true.";
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4732
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4733 static PyObject*
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4734 unicode_splitlines(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4735 {
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4736 int keepends = 0;
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4737
68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4738 if (!PyArg_ParseTuple(args, "\|i:splitlines", &keepends))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4739 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4740
12369 68e48bb94071 Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12329 diff changeset	4741 return PyUnicode_Splitlines((PyObject *)self, keepends);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4742 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4743
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4744 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4745 PyObject unicode_str(PyUnicodeObject self)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4746 {
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	4747 return PyUnicode_AsEncodedString((PyObject *)self, NULL, NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4748 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4749
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4750 static char strip__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4751 "S.strip() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4752 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4753 Return a copy of S with leading and trailing whitespace removed.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4754
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4755 static PyObject *
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4756 unicode_strip(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4757 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4758 return strip(self, 1, 1);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4759 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4760
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4761 static char swapcase__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4762 "S.swapcase() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4763 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4764 Return a copy of S with uppercase characters converted to lowercase\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4765 and vice versa.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4766
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4767 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4768 unicode_swapcase(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4769 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4770 return fixup(self, fixswapcase);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4771 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4772
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4773 static char translate__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4774 "S.translate(table) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4775 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4776 Return a copy of the string S, where all characters have been mapped\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4777 through the given translation table, which must be a mapping of\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4778 Unicode ordinals to Unicode ordinals or None. Unmapped characters\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4779 are left untouched. Characters mapped to None are deleted.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4780
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4781 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4782 unicode_translate(PyUnicodeObject self, PyObject table)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4783 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4784 return PyUnicode_TranslateCharmap(self->str,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4785 self->length,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4786 table,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4787 "ignore");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4788 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4789
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4790 static char upper__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4791 "S.upper() -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4792 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4793 Return a copy of S converted to uppercase.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4794
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4795 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4796 unicode_upper(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4797 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4798 return fixup(self, fixupper);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4799 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4800
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4801 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4802 static char zfill__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4803 "S.zfill(width) -> unicode\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4804 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4805 Pad a numeric string x with zeros on the left, to fill a field\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4806 of the specified width. The string x is never truncated.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4807
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4808 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4809 unicode_zfill(PyUnicodeObject self, PyObject args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4810 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4811 int fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4812 PyUnicodeObject *u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4813
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4814 int width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4815 if (!PyArg_ParseTuple(args, "i:zfill", &width))
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4816 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4817
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4818 if (self->length >= width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4819 Py_INCREF(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4820 return (PyObject*) self;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4821 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4822
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4823 fill = width - self->length;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4824
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4825 u = pad(self, fill, 0, '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4826
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4827 if (u->str[fill] == '+' \|\| u->str[fill] == '-') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4828 /* move sign to beginning of string */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4829 u->str[0] = u->str[fill];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4830 u->str[fill] = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4831 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4832
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4833 return (PyObject*) u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4834 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4835 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4836
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4837 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4838 static PyObject*
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4839 unicode_freelistsize(PyUnicodeObject *self)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4840 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4841 return PyInt_FromLong(unicode_freelist_size);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4842 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4843 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4844
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4845 static char startswith__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4846 "S.startswith(prefix[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4847 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4848 Return 1 if S starts with the specified prefix, otherwise return 0. With\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4849 optional start, test S beginning at that position. With optional end, stop\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4850 comparing S at that position.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4851
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4852 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4853 unicode_startswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4854 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4855 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4856 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4857 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4858 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4859 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4860
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4861 if (!PyArg_ParseTuple(args, "O\|O&O&:startswith", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4862 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4863 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4864 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4865 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4866 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4867 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4868
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4869 result = PyInt_FromLong(tailmatch(self, substring, start, end, -1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4870
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4871 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4872 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4873 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4874
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4875
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4876 static char endswith__doc__[] =
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4877 "S.endswith(suffix[, start[, end]]) -> int\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4878 \n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4879 Return 1 if S ends with the specified suffix, otherwise return 0. With\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4880 optional start, test S beginning at that position. With optional end, stop\n\
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4881 comparing S at that position.";
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4882
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4883 static PyObject *
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4884 unicode_endswith(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4885 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4886 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4887 PyUnicodeObject *substring;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4888 int start = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4889 int end = INT_MAX;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4890 PyObject *result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4891
12629 3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4892 if (!PyArg_ParseTuple(args, "O\|O&O&:endswith", &substring,
3efeaa7521e4 Trent Mick: Guido van Rossum <guido@python.org> parents: 12582 diff changeset	4893 _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end))
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4894 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4895 substring = (PyUnicodeObject *)PyUnicode_FromObject(
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4896 (PyObject *)substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4897 if (substring == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4898 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4899
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4900 result = PyInt_FromLong(tailmatch(self, substring, start, end, +1));
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4901
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4902 Py_DECREF(substring);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4903 return result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4904 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4905
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4906
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4907 static PyMethodDef unicode_methods[] = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4908
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4909 /* Order is according to common usage: often used methods should
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4910 appear first, since lookup is done sequentially. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4911
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4912 {"encode", (PyCFunction) unicode_encode, METH_VARARGS, encode__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4913 {"replace", (PyCFunction) unicode_replace, METH_VARARGS, replace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4914 {"split", (PyCFunction) unicode_split, METH_VARARGS, split__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4915 {"join", (PyCFunction) unicode_join, METH_O, join__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4916 {"capitalize", (PyCFunction) unicode_capitalize, METH_NOARGS, capitalize__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4917 {"title", (PyCFunction) unicode_title, METH_NOARGS, title__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4918 {"center", (PyCFunction) unicode_center, METH_VARARGS, center__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4919 {"count", (PyCFunction) unicode_count, METH_VARARGS, count__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4920 {"expandtabs", (PyCFunction) unicode_expandtabs, METH_VARARGS, expandtabs__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4921 {"find", (PyCFunction) unicode_find, METH_VARARGS, find__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4922 {"index", (PyCFunction) unicode_index, METH_VARARGS, index__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4923 {"ljust", (PyCFunction) unicode_ljust, METH_VARARGS, ljust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4924 {"lower", (PyCFunction) unicode_lower, METH_NOARGS, lower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4925 {"lstrip", (PyCFunction) unicode_lstrip, METH_NOARGS, lstrip__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4926 /* {"maketrans", (PyCFunction) unicode_maketrans, METH_VARARGS, maketrans__doc__}, */
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4927 {"rfind", (PyCFunction) unicode_rfind, METH_VARARGS, rfind__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4928 {"rindex", (PyCFunction) unicode_rindex, METH_VARARGS, rindex__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4929 {"rjust", (PyCFunction) unicode_rjust, METH_VARARGS, rjust__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4930 {"rstrip", (PyCFunction) unicode_rstrip, METH_NOARGS, rstrip__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4931 {"splitlines", (PyCFunction) unicode_splitlines, METH_VARARGS, splitlines__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4932 {"strip", (PyCFunction) unicode_strip, METH_NOARGS, strip__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4933 {"swapcase", (PyCFunction) unicode_swapcase, METH_NOARGS, swapcase__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4934 {"translate", (PyCFunction) unicode_translate, METH_O, translate__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4935 {"upper", (PyCFunction) unicode_upper, METH_NOARGS, upper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4936 {"startswith", (PyCFunction) unicode_startswith, METH_VARARGS, startswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4937 {"endswith", (PyCFunction) unicode_endswith, METH_VARARGS, endswith__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4938 {"islower", (PyCFunction) unicode_islower, METH_NOARGS, islower__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4939 {"isupper", (PyCFunction) unicode_isupper, METH_NOARGS, isupper__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4940 {"istitle", (PyCFunction) unicode_istitle, METH_NOARGS, istitle__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4941 {"isspace", (PyCFunction) unicode_isspace, METH_NOARGS, isspace__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4942 {"isdecimal", (PyCFunction) unicode_isdecimal, METH_NOARGS, isdecimal__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4943 {"isdigit", (PyCFunction) unicode_isdigit, METH_NOARGS, isdigit__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4944 {"isnumeric", (PyCFunction) unicode_isnumeric, METH_NOARGS, isnumeric__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4945 {"isalpha", (PyCFunction) unicode_isalpha, METH_NOARGS, isalpha__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4946 {"isalnum", (PyCFunction) unicode_isalnum, METH_NOARGS, isalnum__doc__},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4947 #if 0
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4948 {"zfill", (PyCFunction) unicode_zfill, METH_VARARGS, zfill__doc__},
b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4949 {"capwords", (PyCFunction) unicode_capwords, METH_NOARGS, capwords__doc__},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4950 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4951
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4952 #if 0
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4953 /* This one is just used for debugging the implementation. */
19301 b9701538f827 Patch #427190: Implement and use METH_NOARGS and METH_O. Martin v. Löwis <martin@v.loewis.de> parents: 19121 diff changeset	4954 {"freelistsize", (PyCFunction) unicode_freelistsize, METH_NOARGS},
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4955 #endif
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4956
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4957 {NULL, NULL}
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4958 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4959
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4960 static PySequenceMethods unicode_as_sequence = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4961 (inquiry) unicode_length, /* sq_length */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4962 (binaryfunc) PyUnicode_Concat, /* sq_concat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4963 (intargfunc) unicode_repeat, /* sq_repeat */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4964 (intargfunc) unicode_getitem, /* sq_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4965 (intintargfunc) unicode_slice, /* sq_slice */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4966 0, /* sq_ass_item */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4967 0, /* sq_ass_slice */
11997 38310eb0659e Add sq_contains implementation. Guido van Rossum <guido@python.org> parents: 11968 diff changeset	4968 (objobjproc)PyUnicode_Contains, /sq_contains/
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4969 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4970
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4971 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4972 unicode_buffer_getreadbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4973 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4974 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4975 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4976 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4977 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4978 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4979 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4980 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4981 ptr = (void ) self->str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4982 return PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4983 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4984
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4985 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4986 unicode_buffer_getwritebuf(PyUnicodeObject *self, int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4987 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4988 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4989 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4990 "cannot use unicode as modifyable buffer");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4991 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4992 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4993
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4994 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4995 unicode_buffer_getsegcount(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4996 int *lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4997 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4998 if (lenp)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	4999 *lenp = PyUnicode_GET_DATA_SIZE(self);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5000 return 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5001 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5002
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5003 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5004 unicode_buffer_getcharbuf(PyUnicodeObject *self,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5005 int index,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5006 const void **ptr)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5007 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5008 PyObject *str;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5009
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5010 if (index != 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5011 PyErr_SetString(PyExc_SystemError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5012 "accessing non-existent unicode segment");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5013 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5014 }
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	5015 str = _PyUnicode_AsDefaultEncodedString((PyObject *)self, NULL);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5016 if (str == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5017 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5018 ptr = (void ) PyString_AS_STRING(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5019 return PyString_GET_SIZE(str);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5020 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5021
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5022 /* Helpers for PyUnicode_Format() */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5023
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5024 static PyObject *
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	5025 getnextarg(PyObject args, int arglen, int p_argidx)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5026 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5027 int argidx = *p_argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5028 if (argidx < arglen) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5029 (*p_argidx)++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5030 if (arglen < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5031 return args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5032 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5033 return PyTuple_GetItem(args, argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5034 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5035 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5036 "not enough arguments for format string");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5037 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5038 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5039
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5040 #define F_LJUST (1<<0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5041 #define F_SIGN (1<<1)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5042 #define F_BLANK (1<<2)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5043 #define F_ALT (1<<3)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5044 #define F_ZERO (1<<4)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5045
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5046 static
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5047 int usprintf(register Py_UNICODE buffer, char format, ...)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5048 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5049 register int i;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5050 int len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5051 va_list va;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5052 char *charbuffer;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5053 va_start(va, format);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5054
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5055 /* First, format the string as char array, then expand to Py_UNICODE
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5056 array. */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5057 charbuffer = (char *)buffer;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5058 len = vsprintf(charbuffer, format, va);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5059 for (i = len - 1; i >= 0; i--)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5060 buffer[i] = (Py_UNICODE) charbuffer[i];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5061
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5062 va_end(va);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5063 return len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5064 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5065
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5066 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5067 formatfloat(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5068 size_t buflen,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5069 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5070 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5071 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5072 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5073 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5074 /* fmt = '%#.' + `prec` + `type`
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5075 worst case length = 3 + 10 (len of INT_MAX) + 1 = 14 (use 20)*/
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5076 char fmt[20];
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5077 double x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5078
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5079 x = PyFloat_AsDouble(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5080 if (x == -1.0 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5081 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5082 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5083 prec = 6;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5084 if (type == 'f' && (fabs(x) / 1e25) >= 1e25)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5085 type = 'g';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5086 sprintf(fmt, "%%%s.%d%c", (flags & F_ALT) ? "#" : "", prec, type);
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5087 /* worst case length calc to ensure no buffer overrun:
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5088 fmt = %#.<prec>g
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5089 buf = '-' + [0-9]*prec + '.' + 'e+' + (longest exp
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5090 for any double rep.)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5091 len = 1 + prec + 1 + 2 + 5 = 9 + prec
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5092 If prec=0 the effective precision is 1 (the leading digit is
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5093 always given), therefore increase by one to 10+prec. */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5094 if (buflen <= (size_t)10 + (size_t)prec) {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5095 PyErr_SetString(PyExc_OverflowError,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5096 "formatted float is too long (precision too long?)");
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5097 return -1;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5098 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5099 return usprintf(buf, fmt, x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5100 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5101
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5102 static PyObject*
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5103 formatlong(PyObject *val, int flags, int prec, int type)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5104 {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5105 char *buf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5106 int i, len;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5107 PyObject str; / temporary string object. */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5108 PyUnicodeObject *result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5109
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5110 str = _PyString_FormatLong(val, flags, prec, type, &buf, &len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5111 if (!str)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5112 return NULL;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5113 result = _PyUnicode_New(len);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5114 for (i = 0; i < len; i++)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5115 result->str[i] = buf[i];
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5116 result->str[len] = 0;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5117 Py_DECREF(str);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5118 return (PyObject*)result;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5119 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5120
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5121 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5122 formatint(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5123 size_t buflen,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5124 int flags,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5125 int prec,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5126 int type,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5127 PyObject *v)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5128 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5129 /* fmt = '%#.' + `prec` + 'l' + `type`
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5130 worst case length = 3 + 19 (worst len of INT_MAX on 64-bit machine)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5131 + 1 + 1 = 24*/
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5132 char fmt[64]; /* plenty big enough! */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5133 long x;
17640 9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5134 int use_native_c_format = 1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5135
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5136 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5137 if (x == -1 && PyErr_Occurred())
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5138 return -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5139 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5140 prec = 1;
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5141 /* buf = '+'/'-'/'0'/'0x' + '[0-9]'*max(prec,len(x in octal))
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5142 worst case buf = '0x' + [0-9]prec, where prec >= 11 /
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5143 if (buflen <= 13 \|\| buflen <= (size_t)2+(size_t)prec) {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5144 PyErr_SetString(PyExc_OverflowError,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5145 "formatted integer is too long (precision too long?)");
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5146 return -1;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5147 }
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5148 /* When converting 0 under %#x or %#X, C leaves off the base marker,
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5149 * but we want it (for consistency with other %#x conversions, and
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5150 * for consistency with Python's hex() function).
17640 9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5151 * BUG 28-Apr-2001 tim: At least two platform Cs (Metrowerks &
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5152 * Compaq Tru64) violate the std by converting 0 w/ leading 0x anyway.
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5153 * So add it only if the platform doesn't already.
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5154 */
17640 9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5155 if (x == 0 && (flags & F_ALT) && (type == 'x' \|\| type == 'X')) {
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5156 /* Only way to know what the platform does is to try it. */
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5157 sprintf(fmt, type == 'x' ? "%#x" : "%#X", 0);
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5158 if (fmt[1] != (char)type) {
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5159 /* Supply our own leading 0x/0X -- needed under std C */
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5160 use_native_c_format = 0;
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5161 sprintf(fmt, "0%c%%#.%dl%c", type, prec, type);
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5162 }
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5163 }
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5164 if (use_native_c_format)
9934225fe297 A different approach to the problem reported in Tim Peters <tim.peters@gmail.com> parents: 17618 diff changeset	5165 sprintf(fmt, "%%%s.%dl%c", (flags & F_ALT) ? "#" : "", prec, type);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5166 return usprintf(buf, fmt, x);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5167 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5168
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5169 static int
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5170 formatchar(Py_UNICODE *buf,
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5171 size_t buflen,
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5172 PyObject *v)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5173 {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5174 /* presume that the buffer is at least 2 characters long */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5175 if (PyUnicode_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5176 if (PyUnicode_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5177 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5178 buf[0] = PyUnicode_AS_UNICODE(v)[0];
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5179 }
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5180
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5181 else if (PyString_Check(v)) {
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5182 if (PyString_GET_SIZE(v) != 1)
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5183 goto onError;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5184 buf[0] = (Py_UNICODE)PyString_AS_STRING(v)[0];
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5185 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5186
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5187 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5188 /* Integer input truncated to a character */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5189 long x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5190 x = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5191 if (x == -1 && PyErr_Occurred())
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5192 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5193 buf[0] = (char) x;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5194 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5195 buf[1] = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5196 return 1;
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5197
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5198 onError:
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5199 PyErr_SetString(PyExc_TypeError,
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5200 "%c requires int or char");
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5201 return -1;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5202 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5203
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5204 /* fmt%(v1,v2,...) is roughly equivalent to sprintf(fmt, v1, v2, ...)
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5205
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5206 FORMATBUFLEN is the length of the buffer in which the floats, ints, &
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5207 chars are formatted. XXX This is a magic number. Each formatting
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5208 routine does bounds checking to ensure no overflow, but a better
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5209 solution may be to malloc a buffer of appropriate size for each
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5210 format. For now, the current solution is sufficient.
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5211 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5212 #define FORMATBUFLEN (size_t)120
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5213
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5214 PyObject PyUnicode_Format(PyObject format,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5215 PyObject *args)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5216 {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5217 Py_UNICODE fmt, res;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5218 int fmtcnt, rescnt, reslen, arglen, argidx;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5219 int args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5220 PyUnicodeObject *result = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5221 PyObject *dict = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5222 PyObject *uformat;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5223
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5224 if (format == NULL \|\| args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5225 PyErr_BadInternalCall();
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5226 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5227 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5228 uformat = PyUnicode_FromObject(format);
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5229 if (uformat == NULL)
c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5230 return NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5231 fmt = PyUnicode_AS_UNICODE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5232 fmtcnt = PyUnicode_GET_SIZE(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5233
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5234 reslen = rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5235 result = _PyUnicode_New(reslen);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5236 if (result == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5237 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5238 res = PyUnicode_AS_UNICODE(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5239
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5240 if (PyTuple_Check(args)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5241 arglen = PyTuple_Size(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5242 argidx = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5243 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5244 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5245 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5246 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5247 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5248 if (args->ob_type->tp_as_mapping)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5249 dict = args;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5250
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5251 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5252 if (*fmt != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5253 if (--rescnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5254 rescnt = fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5255 reslen += rescnt;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5256 if (_PyUnicode_Resize(&result, reslen) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5257 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5258 res = PyUnicode_AS_UNICODE(result) + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5259 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5260 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5261 res++ = fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5262 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5263 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5264 /* Got a format specifier */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5265 int flags = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5266 int width = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5267 int prec = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5268 Py_UNICODE c = '\0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5269 Py_UNICODE fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5270 PyObject *v = NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5271 PyObject *temp = NULL;
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5272 Py_UNICODE *pbuf;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5273 Py_UNICODE sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5274 int len;
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5275 Py_UNICODE formatbuf[FORMATBUFLEN]; /* For format{float,int,char}() */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5276
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5277 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5278 if (*fmt == '(') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5279 Py_UNICODE *keystart;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5280 int keylen;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5281 PyObject *key;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5282 int pcount = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5283
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5284 if (dict == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5285 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5286 "format requires a mapping");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5287 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5288 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5289 ++fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5290 --fmtcnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5291 keystart = fmt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5292 /* Skip over balanced parentheses */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5293 while (pcount > 0 && --fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5294 if (*fmt == ')')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5295 --pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5296 else if (*fmt == '(')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5297 ++pcount;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5298 fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5299 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5300 keylen = fmt - keystart - 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5301 if (fmtcnt < 0 \|\| pcount > 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5302 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5303 "incomplete format key");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5304 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5305 }
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5306 /* keys are converted to strings using UTF-8 and
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5307 then looked up since Python uses strings to hold
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5308 variables names etc. in its namespaces and we
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5309 wouldn't want to break common idioms. */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5310 key = PyUnicode_EncodeUTF8(keystart,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5311 keylen,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5312 NULL);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5313 if (key == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5314 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5315 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5316 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5317 args_owned = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5318 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5319 args = PyObject_GetItem(dict, key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5320 Py_DECREF(key);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5321 if (args == NULL) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5322 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5323 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5324 args_owned = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5325 arglen = -1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5326 argidx = -2;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5327 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5328 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5329 switch (c = *fmt++) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5330 case '-': flags \|= F_LJUST; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5331 case '+': flags \|= F_SIGN; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5332 case ' ': flags \|= F_BLANK; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5333 case '#': flags \|= F_ALT; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5334 case '0': flags \|= F_ZERO; continue;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5335 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5336 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5337 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5338 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5339 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5340 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5341 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5342 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5343 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5344 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5345 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5346 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5347 width = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5348 if (width < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5349 flags \|= F_LJUST;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5350 width = -width;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5351 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5352 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5353 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5354 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5355 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5356 width = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5357 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5358 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5359 if (c < '0' \|\| c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5360 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5361 if ((width*10) / 10 != width) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5362 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5363 "width too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5364 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5365 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5366 width = width*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5367 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5368 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5369 if (c == '.') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5370 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5371 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5372 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5373 if (c == '*') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5374 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5375 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5376 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5377 if (!PyInt_Check(v)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5378 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5379 "* wants int");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5380 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5381 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5382 prec = PyInt_AsLong(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5383 if (prec < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5384 prec = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5385 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5386 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5387 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5388 else if (c >= '0' && c <= '9') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5389 prec = c - '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5390 while (--fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5391 c = Py_CHARMASK(*fmt++);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5392 if (c < '0' \|\| c > '9')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5393 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5394 if ((prec*10) / 10 != prec) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5395 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5396 "prec too big");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5397 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5398 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5399 prec = prec*10 + (c - '0');
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5400 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5401 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5402 } /* prec */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5403 if (fmtcnt >= 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5404 if (c == 'h' \|\| c == 'l' \|\| c == 'L') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5405 if (--fmtcnt >= 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5406 c = *fmt++;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5407 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5408 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5409 if (fmtcnt < 0) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5410 PyErr_SetString(PyExc_ValueError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5411 "incomplete format");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5412 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5413 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5414 if (c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5415 v = getnextarg(args, arglen, &argidx);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5416 if (v == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5417 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5418 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5419 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5420 fill = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5421 switch (c) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5422
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5423 case '%':
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5424 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5425 /* presume that buffer length is at least 1 */
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5426 pbuf[0] = '%';
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5427 len = 1;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5428 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5429
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5430 case 's':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5431 case 'r':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5432 if (PyUnicode_Check(v) && c == 's') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5433 temp = v;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5434 Py_INCREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5435 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5436 else {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5437 PyObject *unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5438 if (c == 's')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5439 temp = PyObject_Str(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5440 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5441 temp = PyObject_Repr(v);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5442 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5443 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5444 if (!PyString_Check(temp)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5445 /* XXX Note: this should never happen, since
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5446 PyObject_Repr() and PyObject_Str() assure
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5447 this */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5448 Py_DECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5449 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5450 "%s argument has non-string str()");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5451 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5452 }
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5453 unicode = PyUnicode_Decode(PyString_AS_STRING(temp),
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5454 PyString_GET_SIZE(temp),
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5455 NULL,
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5456 "strict");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5457 Py_DECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5458 temp = unicode;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5459 if (temp == NULL)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5460 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5461 }
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5462 pbuf = PyUnicode_AS_UNICODE(temp);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5463 len = PyUnicode_GET_SIZE(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5464 if (prec >= 0 && len > prec)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5465 len = prec;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5466 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5467
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5468 case 'i':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5469 case 'd':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5470 case 'u':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5471 case 'o':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5472 case 'x':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5473 case 'X':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5474 if (c == 'i')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5475 c = 'd';
15455 d0146d128897 Fox for SF bug #123859: %[duxXo] long formats inconsistent. Tim Peters <tim.peters@gmail.com> parents: 15001 diff changeset	5476 if (PyLong_Check(v)) {
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5477 temp = formatlong(v, flags, prec, c);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5478 if (!temp)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5479 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5480 pbuf = PyUnicode_AS_UNICODE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5481 len = PyUnicode_GET_SIZE(temp);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5482 /* unbounded ints can always produce
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5483 a sign character! */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5484 sign = 1;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5485 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5486 else {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5487 pbuf = formatbuf;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5488 len = formatint(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5489 flags, prec, c, v);
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5490 if (len < 0)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5491 goto onError;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5492 /* only d conversion is signed */
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5493 sign = c == 'd';
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5494 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5495 if (flags & F_ZERO)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5496 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5497 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5498
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5499 case 'e':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5500 case 'E':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5501 case 'f':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5502 case 'g':
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5503 case 'G':
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5504 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5505 len = formatfloat(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE),
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5506 flags, prec, c, v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5507 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5508 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5509 sign = 1;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5510 if (flags & F_ZERO)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5511 fill = '0';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5512 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5513
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5514 case 'c':
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5515 pbuf = formatbuf;
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5516 len = formatchar(pbuf, sizeof(formatbuf)/sizeof(Py_UNICODE), v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5517 if (len < 0)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5518 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5519 break;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5520
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5521 default:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5522 PyErr_Format(PyExc_ValueError,
15579 31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the Andrew M. Kuchling <amk@amk.ca> parents: 15455 diff changeset	5523 "unsupported format character '%c' (0x%x) "
31d78f361530 [ Patch #102852 ] Make % error a bit more informative by indicates the Andrew M. Kuchling <amk@amk.ca> parents: 15455 diff changeset	5524 "at index %i",
15631 552db8d83acd Patch #102940: use only printable Unicode chars in reporting Andrew M. Kuchling <amk@amk.ca> parents: 15609 diff changeset	5525 (31<=c && c<=126) ? c : '?',
552db8d83acd Patch #102940: use only printable Unicode chars in reporting Andrew M. Kuchling <amk@amk.ca> parents: 15609 diff changeset	5526 c, fmt -1 - PyUnicode_AS_UNICODE(uformat));
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5527 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5528 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5529 if (sign) {
13196 18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5530 if (pbuf == '-' \|\| pbuf == '+') {
18920eaa1226 Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 13110 diff changeset	5531 sign = *pbuf++;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5532 len--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5533 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5534 else if (flags & F_SIGN)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5535 sign = '+';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5536 else if (flags & F_BLANK)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5537 sign = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5538 else
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5539 sign = 0;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5540 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5541 if (width < len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5542 width = len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5543 if (rescnt < width + (sign != 0)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5544 reslen -= rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5545 rescnt = width + fmtcnt + 100;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5546 reslen += rescnt;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5547 if (_PyUnicode_Resize(&result, reslen) < 0)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5548 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5549 res = PyUnicode_AS_UNICODE(result)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5550 + reslen - rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5551 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5552 if (sign) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5553 if (fill != ' ')
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5554 *res++ = sign;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5555 rescnt--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5556 if (width > len)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5557 width--;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5558 }
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5559 if ((flags & F_ALT) && (c == 'x' \|\| c == 'X')) {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5560 assert(pbuf[0] == '0');
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5561 assert(pbuf[1] == c);
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5562 if (fill != ' ') {
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5563 res++ = pbuf++;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5564 res++ = pbuf++;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5565 }
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5566 rescnt -= 2;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5567 width -= 2;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5568 if (width < 0)
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5569 width = 0;
f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5570 len -= 2;
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5571 }
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5572 if (width > len && !(flags & F_LJUST)) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5573 do {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5574 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5575 *res++ = fill;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5576 } while (--width > len);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5577 }
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5578 if (fill == ' ') {
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5579 if (sign)
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5580 *res++ = sign;
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5581 if ((flags & F_ALT) && (c == 'x' \|\| c == 'X')) {
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5582 assert(pbuf[0] == '0');
17450 f74f5aafdc7e Bug 415514 reported that e.g. Tim Peters <tim.peters@gmail.com> parents: 17425 diff changeset	5583 assert(pbuf[1] == c);
14713 58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5584 res++ = pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5585 res++ = pbuf++;
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5586 }
58b832b8745e Derived from Martin's SF patch 110609: support unbounded ints in %d,i,u,x,X,o formats. Tim Peters <tim.peters@gmail.com> parents: 14492 diff changeset	5587 }
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5588 Py_UNICODE_COPY(res, pbuf, len);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5589 res += len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5590 rescnt -= len;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5591 while (--width >= len) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5592 --rescnt;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5593 *res++ = ' ';
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5594 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5595 if (dict && (argidx < arglen) && c != '%') {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5596 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5597 "not all arguments converted");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5598 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5599 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5600 Py_XDECREF(temp);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5601 } /* '%' */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5602 } /* until end */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5603 if (argidx < arglen && !dict) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5604 PyErr_SetString(PyExc_TypeError,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5605 "not all arguments converted");
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5606 goto onError;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5607 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5608
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5609 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5610 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5611 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5612 Py_DECREF(uformat);
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5613 if (_PyUnicode_Resize(&result, reslen - rescnt))
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	5614 goto onError;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5615 return (PyObject *)result;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5616
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5617 onError:
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5618 Py_XDECREF(result);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5619 Py_DECREF(uformat);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5620 if (args_owned) {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5621 Py_DECREF(args);
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5622 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5623 return NULL;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5624 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5625
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5626 static PyBufferProcs unicode_as_buffer = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5627 (getreadbufferproc) unicode_buffer_getreadbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5628 (getwritebufferproc) unicode_buffer_getwritebuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5629 (getsegcountproc) unicode_buffer_getsegcount,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5630 (getcharbufferproc) unicode_buffer_getcharbuf,
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5631 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5632
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5633 staticforward PyObject *
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5634 unicode_subtype_new(PyTypeObject type, PyObject args, PyObject *kwds);
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5635
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5636 static PyObject *
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5637 unicode_new(PyTypeObject type, PyObject args, PyObject *kwds)
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5638 {
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5639 PyObject *x = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5640 static char *kwlist[] = {"string", "encoding", "errors", 0};
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5641 char *encoding = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5642 char *errors = NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5643
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5644 if (type != &PyUnicode_Type)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5645 return unicode_subtype_new(type, args, kwds);
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5646 if (!PyArg_ParseTupleAndKeywords(args, kwds, "\|Oss:unicode",
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5647 kwlist, &x, &encoding, &errors))
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5648 return NULL;
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5649 if (x == NULL)
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5650 return (PyObject *)_PyUnicode_New(0);
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5651 return PyUnicode_FromEncodedObject(x, encoding, errors);
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5652 }
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5653
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5654 static PyObject *
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5655 unicode_subtype_new(PyTypeObject type, PyObject args, PyObject *kwds)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5656 {
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5657 PyUnicodeObject tmp, pnew;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5658 int n;
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5659
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5660 assert(PyType_IsSubtype(type, &PyUnicode_Type));
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5661 tmp = (PyUnicodeObject *)unicode_new(&PyUnicode_Type, args, kwds);
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5662 if (tmp == NULL)
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5663 return NULL;
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5664 assert(PyUnicode_Check(tmp));
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5665 pnew = (PyUnicodeObject *) type->tp_alloc(type, n = tmp->length);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5666 if (pnew == NULL)
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5667 return NULL;
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5668 pnew->str = PyMem_NEW(Py_UNICODE, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5669 if (pnew->str == NULL) {
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5670 _Py_ForgetReference((PyObject *)pnew);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5671 PyObject_DEL(pnew);
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5672 return NULL;
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5673 }
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5674 Py_UNICODE_COPY(pnew->str, tmp->str, n+1);
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5675 pnew->length = n;
3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5676 pnew->hash = tmp->hash;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5677 Py_DECREF(tmp);
19886 3b0da13ace48 str_subtype_new, unicode_subtype_new: Tim Peters <tim.peters@gmail.com> parents: 19885 diff changeset	5678 return (PyObject *)pnew;
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5679 }
a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5680
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5681 static char unicode_doc[] =
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5682 "unicode(string [, encoding[, errors]]) -> object\n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5683 \n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5684 Create a new Unicode object from the given encoded string.\n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5685 encoding defaults to the current default string encoding and \n\
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5686 errors, defining the error handling, to 'strict'.";
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5687
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5688 PyTypeObject PyUnicode_Type = {
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5689 PyObject_HEAD_INIT(&PyType_Type)
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5690 0, /* ob_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5691 "unicode", /* tp_name */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5692 sizeof(PyUnicodeObject), /* tp_size */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5693 0, /* tp_itemsize */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5694 /* Slots */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5695 (destructor)_PyUnicode_Free, /* tp_dealloc */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5696 0, /* tp_print */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5697 0, /* tp_getattr */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5698 0, /* tp_setattr */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5699 (cmpfunc) unicode_compare, /* tp_compare */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5700 (reprfunc) unicode_repr, /* tp_repr */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5701 0, /* tp_as_number */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5702 &unicode_as_sequence, /* tp_as_sequence */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5703 0, /* tp_as_mapping */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5704 (hashfunc) unicode_hash, /* tp_hash*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5705 0, /* tp_call*/
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5706 (reprfunc) unicode_str, /* tp_str */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5707 PyObject_GenericGetAttr, /* tp_getattro */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5708 0, /* tp_setattro */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5709 &unicode_as_buffer, /* tp_as_buffer */
19593 a80b4b006eb7 Make unicode subclassable. Guido van Rossum <guido@python.org> parents: 19301 diff changeset	5710 Py_TPFLAGS_DEFAULT \| Py_TPFLAGS_BASETYPE, /* tp_flags */
18933 09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5711 unicode_doc, /* tp_doc */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5712 0, /* tp_traverse */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5713 0, /* tp_clear */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5714 0, /* tp_richcompare */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5715 0, /* tp_weaklistoffset */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5716 0, /* tp_iter */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5717 0, /* tp_iternext */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5718 unicode_methods, /* tp_methods */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5719 0, /* tp_members */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5720 0, /* tp_getset */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5721 0, /* tp_base */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5722 0, /* tp_dict */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5723 0, /* tp_descr_get */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5724 0, /* tp_descr_set */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5725 0, /* tp_dictoffset */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5726 0, /* tp_init */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5727 0, /* tp_alloc */
09df3254b49d Merge of descr-branch back into trunk. Tim Peters <tim.peters@gmail.com> parents: 18887 diff changeset	5728 unicode_new, /* tp_new */
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5729 };
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5730
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5731 /* Initialize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5732
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	5733 void _PyUnicode_Init(void)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5734 {
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5735 int i;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5736
12645 c4eb70d24a87 M.-A. Lemburg <mal@lemburg.com>: Fred Drake <fdrake@acm.org> parents: 12629 diff changeset	5737 /* Init the implementation */
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5738 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5739 unicode_freelist_size = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5740 unicode_empty = _PyUnicode_New(0);
12876 243593f11fcf Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12646 diff changeset	5741 strcpy(unicode_default_encoding, "ascii");
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5742 for (i = 0; i < 256; i++)
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5743 unicode_latin1[i] = NULL;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5744 }
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5745
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5746 /* Finalize the Unicode implementation */
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5747
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5748 void
13660 a1ce07218781 Miscelaneous ANSIfications. I'm assuming here 'main' should take (int, Thomas Wouters <thomas@python.org> parents: 13635 diff changeset	5749 _PyUnicode_Fini(void)
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5750 {
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	5751 PyUnicodeObject *u;
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5752 int i;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5753
14998 c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free Guido van Rossum <guido@python.org> parents: 14869 diff changeset	5754 Py_XDECREF(unicode_empty);
c16bdf555c79 In _PyUnicode_Fini(), decref unicode_empty before tearng down the free Guido van Rossum <guido@python.org> parents: 14869 diff changeset	5755 unicode_empty = NULL;
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	5756
17618 b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5757 for (i = 0; i < 256; i++) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5758 if (unicode_latin1[i]) {
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5759 Py_DECREF(unicode_latin1[i]);
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5760 unicode_latin1[i] = NULL;
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5761 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5762 }
b44ba7c01217 This patch originated from an idea by Martin v. Loewis who submitted a Marc-André Lemburg <mal@egenix.com> parents: 17600 diff changeset	5763
15001 a349b89bf10e _PyUnicode_Fini(): Initialize the local freelist walking variable `u' Barry Warsaw <barry@python.org> parents: 14998 diff changeset	5764 for (u = unicode_freelist; u != NULL;) {
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5765 PyUnicodeObject *v = u;
13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5766 u = (PyUnicodeObject *)u;
12329 3cc77848a62a Marc-Andre Lemburg: Guido van Rossum <guido@python.org> parents: 12320 diff changeset	5767 if (v->str)
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	5768 PyMem_DEL(v->str);
13837 2badb322355f This patch finalizes the move from UTF-8 to a default encoding in Marc-André Lemburg <mal@egenix.com> parents: 13828 diff changeset	5769 Py_XDECREF(v->defenc);
12576 1c7c2dd1beb1 Vladimir Marangozov's long-awaited malloc restructuring. Guido van Rossum <guido@python.org> parents: 12568 diff changeset	5770 PyObject_DEL(v);
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5771 }
12899 9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5772 unicode_freelist = NULL;
9df91f91e5cc Marc-Andre Lemburg <mal@lemburg.com>: Marc-André Lemburg <mal@egenix.com> parents: 12876 diff changeset	5773 unicode_freelist_size = 0;
11968 13bc68faeb65 Unicode implementation by Marc-Andre Lemburg based on original code by Guido van Rossum <guido@python.org> parents: diff changeset	5774 }

Mercurial > cpython-lru_cache-weakref

annotate Objects/unicodeobject.c @ 20010:d03c19db3094 legacy-trunk